@Christian Bundy

but flume views expect to see each message once, in the order it was received

This seems to suggest that currently, all views are expected to behave like event-streams themselves? My intuition is that while event streams can be a useful abstraction, trying to use them everywhere is probably not the best of ideas. But it's entirely possible I'm misunderstanding encryption and that that view is necessarily a stream.

When I hear "event streams/sourcing" I tend to think of Kleppmann's talk where relational materialized views are important parts of the pipeline. On the other hand everything as a stream reminds me of reactive streams which I think about in an entirely different way. Again, quite possibly in error.

@jaccarmac

Disclaimer: I don't know what I'm talking about, and you're probably better off reading the flumedb readme and the awesome flume-intro video for some more basic info on how the JS implementation handles data.

This seems to suggest that currently, all views are expected to behave like event-streams themselves?

Yeah, the implementation is surprisingly simple:

• The log and views all have sequence numbers that represent their internal state.
• Sequence numbers start at -1 in an empty log / view and increment each time they receive a message.
• The log receives messages, appends to the database, and increments its sequence number.
• The message is run through all transform functions (here be dragons) and then passed to the view.
• The view runs some function on the message (usually map / filter / reduce) and increments its own sequence number.
• You can determine whether a view is completely up-to-date when its sequence number matches the sequence number of the log.

I've written some super simple implementations for both a log and a view if you'd like to see some code, but the gist is views aren't streams but they take streams as input, and currently we don't have a way for a transform to say "hey I just learned to decrypt message 42 and the value should be __" without breaking the semantics behind since. If we add intermediate views we'll need a new way to check whether views are up-to-date. Happy to chat more about this or have a call if you'd like.

@Christian Bundy Thanks for that. I have a hunch that views depending on tracking sequence numbers is an implementation detail causing more trouble than it's worth, but really need to start hacking on SSB things before I say anything else. Won't waste your time with a call as I need to do my reading :)