@Mix Android db2 has transaction support. The messages needs to fit into 1 block (so 65k), but otherwise it is atomic.

@arj that's awesome!

hmmm, 65k is at lowest 8 messages, but could be much higher than that if it's a bunch of small (encrypted) tombstones. Unfortunately, it probably won't help with large csv imports where you have say 400 profiles to make, and 400+ links between them. I'll have to do some benchmarking.

Can I commit a message, get it's message key returned (to be applied within my next message) and then commit the next message all within one transaction?

@nanomonkey That's unfortunately not possible. The input to addTransaction is an array, and we only guarantee atomicity: either all of the messages in the array are written to disk, or none. The input would have to be a function, if you wanted to get the msg key from one of them to plug into the other. There's also no notion of rollback in these transactions.

The input would have to be a function, if you wanted to get the msg key from one of them to plug into the other.

Or a template mechanism could be used - so that the function could be usable over RPC.

proposal: write the transaction set to a file. to get an atomic write, write to filename+~ then move that into position. on start up, check if the transaction file exists. refuse regular writes while transaction is in process. there is no need to disable replication.

I think you'll need to say "start transaction" then "end transaction", with a lock, because you may need to create messages that refer to earlier messages in the transaction, so you need to know the correct hash of that message, which depends on the signature.

So, create all the messages in the transaction, but without writing them, then write them to transaction file, then copy transaction file into the log, then delete transaction file. at startup, check if transaction file exists. check if transaction file contains any messages not in the log, if so write them and delete transaction file.

If crash occurs after writing transaction file, it's recoverable, if it fails before writing transaction file it's not. Queue any regular writes until after transation file is deleted.