You are reading content from Scuttlebutt
@andrestaltz %6lndYneuKsH4IumtVr+uQwDPejvkozuMHLxEAegtA+0=.sha256
Re: %b6nlgiAu3

Update.

I've been struggling and doing slow progress mostly due to the feedback cycle being quite slow, I have to nuke node_modules and restart the app often.

I rewrote asyncstorage-down to be compatible with the latest abstract-leveldown, got all tests passing. But hit some problems because some db keys were intarrays and the way gt/lt/lte/gte ranges was implemented in asyncstorage-down had lexicographic comparison of keys, and I don't think that quite works with intarrays (it works well with keys). So I basically disabled gt/lt/lte/gte in asyncstorage-down, which made it not anymore pass all abstract-leveldown tests, but at least I got it working for sbot on android.

Next problem I hit, which I'm currently trying to solve is related to logDB. In secure-scuttlebutt v15 (not the flume one), we have this logic for createLogStream:

 db.createLogStream = Limit(Live(function (opts) {
    opts = stdopts(opts)
    var keys = opts.keys; delete opts.keys
    var values = opts.values; delete opts.values
    return pull(
      pl.old(logDB, stdopts(opts)),
      //lookup2(keys, values, 'timestamp')
      paramap(function (data, cb) {
        var key = data.value
        var seq = data.key
        db.get(key, function (err, value) {
          if (err) cb(err)
          else cb(null, msgFmt(keys, values, {key: key, value: value, timestamp: seq}))
        })
      })
    )
  }, function (opts) {
    return pl.live(db, stdopts(opts))
  }))

The important part is var key = data.value and db.get(key, .... I'm hitting the situation where data.value is actually an ssb message, like {author, content, sequence, timestamp} etc. It's an object. And down the stack below db.get(), bytewise.encode will fail to encode that object.

So currently I'm trying to figure out why does pl.old(logDB, stdopts(opts)) serve SSB messages under data.value. I believe data.value should be here some valid SSB ref.

User has not chosen to be hosted publicly
@Dominic %WxZTYnlJ3xbQs9nEjAdp4TzVK1ZptDjos1dVcz6ybVo=.sha256

yes, logDB should be local_timestamp -> hash(msg)

What happens if you just dump logDB on it's own? does it also output keys correctly and then objects? or does it always output objects (if so, check the point where they get written)

Okay, I think what is happening: secure-scuttlebutt depends on level-sublevel, and log db is actually part of the main ssb db, https://github.com/ssbc/secure-scuttlebutt/blob/6dde1c5e1d32a7427848bfad8a4641577bd0d1ab/index.js#L91-L109
since you disabled ranges, it's breaking this and you've just read past the range that logDB is located in, into the rest of the database - where the keys are hashes and the values are objects.

@andrestaltz %2Jq120/701j6abRSGCXEWeJKP8ynLkCxmp4KPmcxLOA=.sha256

Okay, I think what is happening: secure-scuttlebutt depends on level-sublevel, and log db is actually part of the main ssb db, https://github.com/ssbc/secure-scuttlebutt/blob/6dde1c5e1d32a7427848bfad8a4641577bd0d1ab/index.js#L91-L109
since you disabled ranges, it's breaking this and you've just read past the range that logDB is located in, into the rest of the database - where the keys are hashes and the values are objects.

That seem like an exact diagnosis of my problem. I guess I just need to carefully implement ranges and lexicographic comparison of UInt8Arrays (maybe a naive key.toString('hex') would work).

Join Scuttlebutt now