You are reading content from Scuttlebutt
@andrestaltz %UOHPkyucRQBbgt3JbCgrH/d61vzWVo2KtmtWfWHK1sw=.sha256
Re: %fV0IabUtn


I thought this was ready, so I started doing final tests in Manyverse, using my production database (offline, of course, so no one would replicate). Deleting feeds that are blocked, then compacting, then reindexing. I want to make sure delete-compact-reindex is rock solid, because I hate debugging mysterious crash reports from users, and users also hate crashes of course. Better safe and neurotic than sorry.

The performance is overall acceptable, and the recent boosts in performance in ssb-db2 certainly help the compaction use case.

However, I ran into bugs. Nasty ones: crashes during post-compaction reindexing. Took me approximately one day to debug the crash, and the fix was somewhat straightforward after that:

I also predicted a possible bug (but didn't reproduce it), and began working on a fix for that possible bug:

However, I ran into yet another nasty case. If there are deleted msgs in the log, jitdb queries will either give the wrong size of paginated results, or give duplicate results, or both. This has nothing to do with compaction, it's actually an existing risk in production that anyone has if there are deleted msgs in their log, which I believe so far is no one! We're lucky with that. But the bug has to be fixed. I reported it here and will think about a proper solution:

Join Scuttlebutt now