I'm not 100% happy with the state of ebt but it's not in a bad state. I will copy the recent comments from the github PR here:
- Verifies incoming messages
- Stores vector clocks of each peer it talked to and updates them
- Streams messages to peers live
- Only send messages of feeds that a peer wants (receive bit on the note)
- Filter by feed type -just disabled everything that isnt current ssb for now
1) Update the vector clock / want list not just once after connecting
Currently the sets of feeds we want to receive is just send once when the connection is established.
The JS documentation is a bit lacking about when to actually re-send those and thus confirm received messages.
But also it doesn't send a clock update once we call
sbot.Replicate(feed) - which means reception is delayed until a reconnect... at least this a part should be fixed before merging this.
1) implement a more scalable solution to canceling individual feed subscriptions
Right now there is a context per feed. This works but is inefficient. Probably need to re-work the multisink so that they can be unsubscribed directly.
1) Partition sets of feeds over multiple connections
One of the advantages of ebt, other then not asking about feeds that didn't change, is receiving feeds from the best known source. For this we need a bit of heuristics, like ping to the peer and who has messages sooner.
Preliminary testing results: functional but a couple of glitches.
rebuilding of the own/self network frontier after changes on the replication lister (graph walk)
The TODO above about this acknowledged a slight variation of this but it's actually worse. Right now, it just builds the network frontier for ebt when it doesn't have one.. any (un)follows after the first ebt connection are without effect.
Followgraph walk finishes to soon
Tangential and slightly sure it was the case before these changes. The graph/builder walk along the follow messages sometimes collapses and returns to few results.
Untangle circular dependency between state matrix and graph builder
Before the graph builder was hooked up after the indexing system.
The way it's setup now, the combined index needs the state matrix to update it with new received messages... which also needs the graph now for the first fill.
This circle needs to be broken so that the server setup doesn't race.
(the mentioned deadlock glitch turned out to be a muxrpc problem, unrelated to EBT but related to the missing ACK and how to decide if this stream will error problem I mentioned in the previous.)