@arj Benchmark script, the times below are for (de)serializing my complete feed. For deserializing, the output is immediately thrown away, there's not even memory allocation (else, clmr would outperform the json even more).

Quick summary (full cargo bench output at the end of this post):

• parse full feed from compact json: 38.134 ms

• parse full feed from clmr: 5.1947 ms

• serialize full feed as compact json: 1.2881 ms

• serialize full feed as clmr: 396.65 us

No guarantees that the clmr implementation is bug-free, but at least serializing and then deserializing the feed as clmr results in the exact same feed, so that's a good indicator that the benchmark does not rely on buggy behavior.

That was more drastic than I expected. I think the biggest gain is not having to base64 encode/decode everything. The json implementation decodes all base64 into actual byte buffers. Since that needs to happen at some point anyways, and it is most efficient to just do it once during deserialization, I think it is fair to include this in the benchmark.

There also a few more advantages not captured in this benchmark:

• smaller encoding sizes lead to less i/o and memory allocation overall
• can pass around pointers into the raw message (in particular to the raw signature and author) rather than having to copy memory

CC @Piet, @dinosaur, @Dominic

cargo bench

Benchmarking deserialize each message/deserialize signing json: Collecting 100 samples in estimated 194.08 s (5050 iteration                                                                                                                            deserialize each message/deserialize signing json                        
                        time:   [38.377 ms 38.386 ms 38.398 ms]
                        change: [-0.9875% -0.9166% -0.8528%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  6 (6.00%) high mild
  2 (2.00%) high severe
Benchmarking deserialize each message/deserialize compact json: Collecting 100 samples in estimated 193.00 s (5050 iteration                                                                                                                            deserialize each message/deserialize compact json                        
                        time:   [38.128 ms 38.134 ms 38.140 ms]
                        change: [-1.0764% -1.0070% -0.9280%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe
deserialize each message/deserialize clmr                                                                             
                        time:   [5.1803 ms 5.1947 ms 5.2089 ms]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

serialize each message/serialize signing json                                                                             
                        time:   [1.2791 ms 1.2829 ms 1.2872 ms]
Found 7 outliers among 100 measurements (7.00%)
  7 (7.00%) high mild
serialize each message/serialize compact json                                                                             
                        time:   [1.2847 ms 1.2881 ms 1.2921 ms]
Found 12 outliers among 100 measurements (12.00%)
  6 (6.00%) high mild
  6 (6.00%) high severe
serialize each message/serialize clmr                                                                            
                        time:   [395.12 us 396.65 us 398.28 us]
Found 16 outliers among 100 measurements (16.00%)
  2 (2.00%) low mild
  6 (6.00%) high mild
  8 (8.00%) high severe