You are reading content from Scuttlebutt
@cryptix %ZYwmPHNdXXhzgskG5Aticp1YRMiB22DEKZHEviik0/c=.sha256
Re: %i6pDs9WVW

At this point you might ask, but why did you say "no" to usable then?

The answer and unsolved problem lies in the timestamp sorting I convinced myself I had to add to this to have a usable whole database. While this is functional there are a few bugs around nulling and re-fetching feeds, which happens if you block someone for instance. I haven't been able to reproduce in tests yet.

These bugs might cripple your queries until you delete the index folder manually for re-indexing and putting it out there in this state seemed like the opposite of what I tried to shot for (a usable database for clients). So I'm haunted a bit by my want for too much and stretched by the need to get started with EBT.

OTOH this feature was also needed for partial replication, so it seemed like a given to just do it. Not the timestamp (we all agree claimed ts is evil, of course) but the feed sequence itself. Once feeds are saved out-of-order in the receive log we need a way to assembled their intended order again.

@arj shared his solution for jitdb with me and I must say it's glaringly simple and awesome fast. For each entry in the receive offset log you hold the value you want to sort or paginate by in a gigantic array and serialize that to and from disk on startup. I implemented three domains for now: claimed ts, received ts and feed sequence.

I think and hope I just messed up around the serialization somewhere and then it hits a consistency failure case (len(rxlog) != len(seqsorter)).

I hope to get this done done once I have made some headway on EBT but right now there is too much ground to cover for trailing around this for another week.

Join Scuttlebutt now