Update

Okay, I executed a surgery on jitdb (PR #220), and now updating or creating indexes with it is faster.

But something is still slow with reindexing-after-compacting. Like in my tests, initial indexing is almost 2x faster than post-compaction reindexing. I took a deep look with the Chrome profiler, and discovered something disgusting. I shared it with arj and we laughed.

When you call level.clear() (e.g. when a leveldb index is being reset prior to reindexing), it does the dumbest operation it could possibly do: iterate (in JS, not in C) on each item and call "delete" on each. Proof here. So we're going to have roll our own "clear()" which does something more nuclear: delete the leveldb folder using FS APIs, and recreate the leveldb instance.

I feel like I'm getting good at building databases when I see other database implementations doing worse.