%kKbJZkEg0jS8oasX9cJn5JZ+sOQ6WQdMdsWyDqRXqAI=.sha256@noffle. I made a flume index for chess games here: https://github.com/Happy0/ssb-chess-db (it's not the most efficient data structure, but it'll do for now since there are relatively little chess games :P.) I thought I'd quickly write down what I know from having done it. Maybe I could write a tutorial where I take the time to make sure it's understandable, but for now we can just have a conversation about it :P.
Note: my index is 'in memory' using flumeview-reduce ( https://github.com/flumedb/flumeview-reduce ). I'm not sure how to keep it on disk (it's something I was also confused about when I started building an index for chess games and I also came across the current docs thing you linked to and didn't understand it, so I decided to just do it in memory using flumeview-reduce for now.)
Perhaps flumeview-reduce will work for your use case, so I'll write what I know about it (including scuttlebot plugins which you probably already know a bit about):
A scuttlebutt plugin should expose an 'init' function that accepts 'ssb' and 'config' params. ssb is the scuttlebot instance, and config is its config. The init function should return an object of functions that can be invoked on the plugin via the scuttlebot instance (via mux-rpc).
The plugin should expose a manifest via a exports.manifest file that describes the mux-rpc (https://github.com/ssbc/muxrpc) functions that your plugin exposes and their return types. See the mux-rpc github page for more details of what those return types can be.
It should also expose a plugin name and a version. There are some gotchas about naming your plugin that I don't really understand. More details about my confusion in this thread: %fYE4sen... . Since @arj found a workaround and documented it on patchaby's github page, I moved on from investigating it. Maybe I could return to it (especially if I do try to write a tutorial.)
Those functions can then read the view created by the index and process it in useful ways.
If you're building an in-memory index, you should call sbot._flumeUse, give it the name of the index as the first parameter, and as the second param a flumeview-reduce object with a mapper and a reduce (i guess there may be other things you can pass to flumeUse that would use the disk). The mapper and reducer you supply are called for each message in your scuttlebot database (and newly arriving ones), and your reducer should build up your index (in my case, a dictionary of chess games with a game ID as a key and the game status (invited, playing, ended)).. _flumeUse returns a 'viewobject, which you can callview.get` on to get the current state of your index as a callback.
The index is written to the disk periodically, so next time you start scuttlebot it will start processing messages from the last one it has processed. If you change the layout of your index, you can bump the version param given to flumeview-reduce and it will process all the messages in your database from scratch again.
