Flumeview-level

a first pass sketch

Flume views of the level type are like an old school library catalogue system. Messages are passed to the view as the come in, and you make an index which is made up of "lookup cards" ... e.g. for each message you look for animals mentioned in that message then make a card which says:

• which animal word
• who said it
• when it was said
  AND leave a pointer to where that original message was (e.g. means this was said in message 1 in the system, you can find the original content there).

Like a physical library catalogue, these index cards take up way less space and are super quick to skin across compared to walking around the whole Library of Scuttlevillia opening every book.

So once you have this catalogue (view/ index) that tracks animals mentioned, you can then ask questions like "yo I want to see all messages that mention a cat in the last 6 months".
The view quick-snap looks across your index (catalogue of index cards) and is like "oh this cards matches that query... and this one.. and this one" (a stream of cards comes out). Your scuttlebot then goes "Let me fetch each of those messages off the shelf for you" and converts that stream of little cards into a stream of messages.
BOOM

You can extend this story by adding:

• examples of good uses for flumeview-reduce
• what the type of view excells at / is weak at
• exmaples of where it's used right now

#flume #flumedb

cc @aljoscha