Okay, got this together enough to run against the benchmark I used for flumelog-offset. Following are the result of using it with raw buffers (no json).

results for flumelog-offset:

name, ops/second, mb/second, ops, total-mb, seconds
append, 119387.461, 15.1, 1193994, 151.021, 10.001
stream, 325871.724, 41.217, 1193994, 151.021, 3.664
stream no cache, 427648.28, 54.09, 1193994, 151.021, 2.792
stream10, 421363.363, 53.295, 4218690, 533.597, 10.012
random, 32891.11, 4.16, 328944, 41.606, 10.001

in particular, it can read back the stream at 41 mb/second

and here are the results for, flumelog-random-access-file:

name, ops/second, mb/second, ops, total-mb, seconds
append, 147213.757, 18.623, 1472432, 186.272, 10.002
stream, 1084265.095, 137.166, 1472432, 186.272, 1.358
stream no cache, 1505554.192, 190.462, 1472432, 186.272, 0.978
stream10, 1904335.23, 240.91, 14724320, 1862.721, 7.732
random, 447818.718, 56.652, 4478635, 566.578, 10.001

41 becomes 137! over 3x faster! stream10 is 5 times faster, and reading random records is over 10x!

Using JSON the difference isn't as big... but still good, 16mb/s streaming for -offset, and 25mb/s streaming for -random-access-file.

So, I want to use this with an in-place format, and see how much difference it makes once the whole application is running like that.