You are reading content from Scuttlebutt
@andrestaltz %s1rlzYaCHEYaeMWzCaDAPfTFL9PWtgFO6GBPrhIDl3Q=.sha256
Re: %fV0IabUtn


Okay, I executed a surgery on jitdb (PR #220), and now updating or creating indexes with it is faster.

But something is still slow with reindexing-after-compacting. Like in my tests, initial indexing is almost 2x faster than post-compaction reindexing. I took a deep look with the Chrome profiler, and discovered something disgusting. I shared it with arj and we laughed.

When you call level.clear() (e.g. when a leveldb index is being reset prior to reindexing), it does the dumbest operation it could possibly do: iterate (in JS, not in C) on each item and call "delete" on each. Proof here. So we're going to have roll our own "clear()" which does something more nuclear: delete the leveldb folder using FS APIs, and recreate the leveldb instance.

I feel like I'm getting good at building databases when I see other database implementations doing worse. :sweat_smile:

Join Scuttlebutt now