Dev Diary benchmark ci grant:

2018-01-17

Watched the excellent flume intro
video and that
got me really exciting how flume might improve performance in general
(%KiSCJImPPNECdvHLTaftQTie0HnRrCIx9KzQndgrD5c=.sha256).

Decided to start with looking at ssb-viewer (refactor branch) since
that is one of the older code bases.

Some observations from mucking around in the
bench-ssb-viewer repo:

High variation in the running time. Doing multiple runs speeds up the
average time considerably.

Just starting node seems to put a baseline 2-300ms delay. Ssb-client
600ms.

Got the time it takes to render a channel of around 200 messages
down from 2.8s to around 2s by using ssb-about. It seems that patchfoo
uses similar logic for about. Patchcore and patchless both uses
flume. For patchcore it seems to load everything into memory twice as
it needs the data to be wrapped in mutant. Wondering if that could be
made lazy.

Looking at how about messages are stored in flume view:

total 4534 ~1.7MB.

• about on @ people 4109
• other 425 (~10%)

other comes in the following form:

• gatherings 125
• books 21
• git 273

I'm starting to think that maybe about messages in they current form
is an anti-pattern, a relic of a pre-flume history, given how easy it
is to write your own indexes.

I meant to say that it is an anti-pattern for applications.