Open Question: How to disable private message indexing in secure-scuttlebutt?

Is it possible to disable private message indexing in secure-scuttlebutt?

I want to only decrypt private messages at the moment that they are viewed, but secure-scuttlebutt^18 automatically decrypts a scuttlebot's private messages and stores them in my .ssb/flume directory.

I do not want to store these messages in plain text.

Does anyone know if it is possible to easily disable this functionality?

Let's move discussion about this security flaw over here, where I can work on this in public.

Private messages also leak in Decent, here's a screenshot:

I think the solution here is to not store private messages in plain text, as they are then made available over muxrpc.

I don't think this is directly exposed in non-liteclients. But, what's stopping someone from pretending to be a lite client and muxrpcing into your local client if they can figure out where your local sbot is?

Maybe this should be posted to Github by someone who is not blocked by Dominic? He won't see this if I post it to Github, because he blocks me on Github -- and he also deletes my comments.

How big of a security risk is this?

Consider if you're in a room full of experienced ssb programmers, all on the same wifi.

Someone could write a script that attacks your ssb server, simply by running an ssb query over muxrpc directed at your public key -- this query will return your private messages.

So yes, this is a high priority to fix this, for me. But, I'm only on wifi with @gb, and I trust her not to make this request to my local sbot.

Ok, so I haven't fixed this yet, mostly because of my compete unfamiliarity with the secure-scuttlebutt module.

I should also note that I'm using a fork of scuttlebot from lts, so I'm not exactly using the latest ssb either. The connections layer doesn't work for me, so I can't use the last three major version of ssb over here.

I tried just commenting out the unboxer stuff, and that didn't work because errors started throwing in all kinds of flume modules that I have no control over.

My first step has been getting more familiar with how secure-scuttlebutt works, and how it has changed over time. In the process I did discover when this vulnerability was introduced:

One idea I have is to just go back to a version of scuttlebot before March 5th 2018. Then I won't have the connections layer or decrypted private message indexes.

I just hit a wall, because I tried downloading and npm installing scuttlebot@10.5.2https://github.com/ssbc/ssb-server/commit/0f633f3396be68d6207ec3d2fd1d1f180f2a4662 because I'm on a public wifi connection that's not letting me download leveldb prebuilds for some reason -- and this version depends on an older version of leveldb than I have on my system right now.

I'm also a little zonked because of reading a lot of code that I haven't previously read before. I'm taking a break.

Someone could write a script that attacks your ssb server, simply by running an ssb query over muxrpc directed at your public key -- this query will return your private messages.

I don't get it. This should only happen if you run noauth on an address that is dialable from the outside. Otherwise these methods shouldn't be exposed in the first place.

why are some messages leaking while others seem to remain private? - @punkmonk.termux

You're seeing the private messages that are unboxed on the server. Because the pub only has access to it's own private key, it would be unable to unbox messages from other people.

it might be useful to know who this might impact. is it accurate to say that only private messages that were posted via a public facing web client? maybe there could be scenarios where that wouldn't be the case, but that seems to be the pattern and wanted to verify. - %ytjlIM4...

ssb-ws allows sbot.query requests out of the box, so if you have websockets enabled all a person would need to do is run an ssb-ws query directed at your public key in order to request your private messages over the wire.

I think if you're behind NAT it'd be pretty hard for someone to request your private messages over ws. However, if you're sharing wifi with someone who knows how to do this, it'd be pretty easy to do this. It'd also be easy to do this to a pub, or anyone who's connecting via cjdns (or not mesh networking software).

It could be that the connections layer that @arj implemented would protect against this attack, as long as you've not enabled websockets.

But I wonder if it'd be possible to do this over tcp/ip connections too? Those are enabled by default, as far as I know.

But we also gotta keep in mind that the connections layer was implemented in September 2018, and this vulnerability was first introduced in March 2018.

The thing that is really concerning to me is that no one who hasn't been kicked out of the ssbc has responded to this security vulnerability yet (or opened an issue on Github). It'd be good to know whether or not this effects the larger group of people who are using Patchwork.

I'm not using private messages in a way that I'd be embarrassed or my security would be compromised if someone had access to them. In theory all cryptography will be broken, someday, so I'm not sending things I wouldn't want someone to read.

However, if I'd used Dark Crystal to back up a cryptocurrency wallet private key, I'd be moving those coins about right now to a new private key.

It's impossible to know if anyone knew how to use this attack to fetch private messages, but better safe than sorry in my opinion.

Boosting: %zRYGPCI...

I just hit a wall, because I tried downloading and npm installing scuttlebot@10.5.2https://github.com/ssbc/ssb-server/commit/0f633f3396be68d6207ec3d2fd1d1f180f2a4662 because I'm on a public wifi connection that's not letting me download leveldb prebuilds for some reason -- and this version depends on an older version of leveldb than I have on my system right now.

Update: I'm unable to get older versions of scuttlebot to build on my machine because of leveldown build errors... So I guess I'm stuck working with minsbot [Github] and scuttlebot@latest [Github] as I try to fix this.

Update: I tried the approach @Christian Bundy recommended above in ssb-server@14.1.3:

  const unboxerMap = (msg, cb) => cb(null, msg)
  //const unboxerMap = (msg, cb) => cb(null, db.unbox(msg))

And regenerated indexes, but I was still able to view my private messages without my key present in the lite client.

Maybe I was doing something wrong, or this isn't quite the fix. I'm not quite sure yet.

Update: I'm going to give this a rest and go get some groceries.

I just want to say that it's kind of weird that no one has responded to this from the ssbc, even though a few people at scuttlecamp are obviously able to get Internet, and thus they've read these messages.

A simple: "yah, we're gonna fix that." or "we're working on fixing it right now." would do a lot to calm this down.

Is there anyone at scuttlecamp who has asked Dominic about how to fix this?

I mean clearly this should be fixed, right?

Anyway, groceries and then I'll bang my head against this more tonight. The good news is I'm getting very familiar with the core secure-scuttlebutt module where all of the boxing/unboxing happens. I hadn't really looked at it until now, because it worked!

@ev, patchfoo says that you are

Blocked by

@Dash₁ @Zach₁ @jacob₁ @Piet₁ @happy0₁ @andrestaltz₁ @dan₁ @kawaiipunk₁ @william₁ @glyph₁ @dominic₁ @pub_dominic₁ @james₁ @dinosaur₁ @alanna₁ @Rich₁ @andrestaltz_phone₁ (17)

There are several from SSBC in that group, aren't there? So perhaps nobody from SSBC have noticed your posts. Or they're busy at the Scuttle Camp.

I will see if I can make a writeup tomorrow, but it's quite inconvenient if messages from you to @dominic has to go though me. I know nothing about the scuttlebot code. I am as interested as anyone to have the bug that allows anyone to see somebody else's private messages – including dark crystal shards – fixed, but I'm not convinced that I'm the best messenger for this.

I did some experiments today against my pub and it appears to not reveal private messages over the tcp/ip sockets. My pub is running ssb-server dc13e5662776a4faa1cd2c7e128d743e72e60885(shortly after v14.0.4).

1. If I ask my local server I can see the message contents(Intentional I believe)

$ sbotc query.read '{"query": [{"$filter": {"key": "%hMiisUGk02zL60fNkjtVnWjtYUQ73Dr5ux5Q5mHClo0=.sha256"}}]}'

{"key":"%hMiisUGk02zL60fNkjtVnWjtYUQ73Dr5ux5Q5mHClo0=.sha256","value":{..."text":"Test private message to my pub","recps":["@PZjQKGbh9KrWHAp1lo9nH/WXqGLp5rpqpVYLccJLkE4=.ed25519","@6lOh+rLq4MQubPQoKenbB9ipq75NVc2FAjy30MTc08o=.ed25519"]...

1. If I ask my pub it returns the wrapped version to createFeedStream and rejects query.read

$ sbotc -k '@6lOh+rLq4MQubPQoKenbB9ipq75NVc2FAjy30MTc08o=.ed25519' -s scuttlepi.local createHistoryStream '{"id":"@PZjQKGbh9KrWHAp1lo9nH/WXqGLp5rpqpVYLccJLkE4=.ed25519","seq":867}'

{"key":"%hMiisUGk02zL60fNkjtVnWjtYUQ73Dr5ux5Q5mHClo0=.sha256","value":{..."content":"G9OSu8eZJ0+GJP...


$ sbotc -k '@6lOh+rLq4MQubPQoKenbB9ipq75NVc2FAjy30MTc08o=.ed25519' -s scuttlepi.local query.read '{"query": [{"$filter": {"key": "%hMiisUGk02zL60fNkjtVnWjtYUQ73Dr5ux5Q5mHClo0=.sha256"}}]}'

{"message":"method:query,read is not in list of allowed methods","name":"Error","stack":"..."}

Based on this information it seems likely that the data is only exposed on the ws connection. Might be worth disabling that or firewalling it if you are worried about private message contents until this is fixed.

@ev

a few people at scuttlecamp are obviously able to get Internet, and thus they've read these messages

I won't speak for others, but personally I've been offline and haven't seen your messages. Our internet access is extremely limited in both bandwidth and capacity, and most of us have been spending time face-to-face.

I'd love if you could unpack this issue a bit. Could you post the exact configuration that allows LAN peers to query ssb-ws? My intuition is that this would require ssb-ws to be:

• enabled
• over the network
• with noauth

For example, an ~/.ssb/config:

{
  "connections": {
    "incoming": {
      "ws": [{
        "scope": ["public"],
        "port": 9000,
        "transform": "noauth"
      }]
    }
  }
}

I'd be surprised if this information was being leaked with ssb-ws disabled, or with a seemingly secure config like:

{
  "connections": {
    "incoming": {
      "ws": [{
        "scope": ["device"],
        "port": 9000,
        "transform": "shs"
      }]
    }
  }
}

@ev you must have really pissed people off to have everyone block you like that. None of my business really, but you may want to change your game plan a bit, "plays well with others" is one of those sandbox skills you need to have to get by. Just sayin....

peace on earth good will towards bears - @moid

@moid I know why many of these people are blocking me: It's because I came of age in Bloomberg-era New York where the subway system was constantly telling me: "If you see something, say something." I have to balance that with growing up in (and returning to) Chicago where the saying goes: "Snitches get stitches."

As for sandbox skills: I've never wanted to play in a sandbox with children who are intolerant of other children who are different -- or have different opinions -- than the other children. So between New York and Chicago, I'll continue to advocate for inclusive (and secure) sandboxes for us to play in.

I've been bending over backwards not to accuse anyone of doing this on purpose, as there's no reason to believe that right now. But, if we're using (or have used) insecure-scuttlebutt I'm not going to shut up about it.

@ev just open an issue in GH, especially if you have a reproducible test case and/or can point to the relevant code. Someone will see it there - @moid

After the event-stream debacle, I'm not quite sure if opening a Github issue in full public view of the entire development community is the most responsible way to handle this security vulnerability. But if people who are not kicked out of the ssbc want to bring this up on Github, then they should.

@ev

I'd really like to get this issue confirmed either way. If you have a reproducible test case or could post your config, that would be super helpful. For example, if you're using "noauth" with the interface scoped to the network then it sounds like that would cause the issue you're describing. From the ssb-ws readme:

noauth

given the flexibility of multiserver, you may want to run
this with noauth config locally. However this is not recommended
until ssb-ws prevents dns rebinding attacks
and websocket connections from locally open websites.

If used with a secure transport,
that authenticates the client "transport": "shs"
then this is not a problem.

I won't speak for others, but personally I've been offline and haven't seen your messages. Our internet access is extremely limited in both bandwidth and capacity, and most of us have been spending time face-to-face. - @Christian Bundy

Makes sense!

I'd love if you could unpack this issue a bit. Could you post the exact configuration that allows LAN peers to query ssb-ws? - @Christian Bundy

I'm using a fork of the lts version of scuttlebot that doesn't have the connections layer. I've had a difficult time getting the lite client to work with the connections layer in a reliable way.

If the connections layer solves this issue, that's good news, as long as you only allow your device to connect to the scuttlebot.

As @DavidW mentioned, private messages do not appear to be exposed over tcp/ip, which is also good news.

It could be this is just an issue with enabling global websockets, in which case maybe this is just a problem with the lite client(s)?

@Christian Bundy Sorry, our messages passed each other in cypherspace

If used with a secure transport, that authenticates the client "transport": "shs" then this is not a problem.

I am using shs to connect using the lite client, but I'm not using the connections layer over here.

@bundy, you can see the result at http://ssblist.com/: If you choose the ‘private’ tab you will be able to read private messages between accounts that are not yours.

See also %iFLUeZ9….

@ev

I'm using a fork of the lts version of scuttlebot that doesn't have the connections layer

Is this bug specific to scuttlebot@11.4.3, or your fork? I've skimmed mvd and it looks like you're also using a thing called decent-ws rather than ssb-ws, so I want to make sure we're on the same page about which software this affects and which configuration is required to trigger it.

It could be this is just an issue with enabling global websockets

Could you specify what this means? The stack has changed quite a bit in the past six months, so I want to make sure that we're talking about exactly the same thing. Sorry for all of the questions, it's just that I'm struggling to understand and reproduce the issue without having all of the details. I can see the bug being triggered on a website, but I can't find the source for that website to identify what's going on.

Could you specify what this means? The stack has changed quite a bit in the past six months, so I want to make sure that we're talking about exactly the same thing. Sorry for all of the questions, it's just that I'm struggling to understand and reproduce the issue without having all of the details. I can see the bug being triggered on a website, but I can't find the source for that website to identify what's going on. - @Christian Bundy

Thanks for taking the time to look at this, it's helpful to have other opinions about what's going on here.

You're correct that the stack has changed a lot in the last six months, and I've had trouble keeping up because many of the changes haven't been explained or documented very well. I had a lot of trouble getting the network layer to work with the lite client, so ended reverting to scuttlebot @ lts.

If this doesn't effect the latest scuttlebot, then it'd be great to know if this issue only effects my clients and (I think) @regular's clients.

You're correct, I'm using decent-ws, which lets anyone connect to http://decent.evbogue.com/ or http://ssblist.com/ -- which would mean wider exposure than ssb-ws, which only allows your friends to connect with you via websockets.

We can go deeper into the design decisions behind decent-ws vs ssb-ws, and I understand that decent-ws is probably not supported by the ssbc.

github for ease of navigation: https://github.com/evbogue/decent-ws and here is the exact code that is running on my servers: https://github.com/evbogue/minsbot

My concern is that this vulnerability may have exposed private messages to friends between March 2018 and September 2018 when the network layer was implemented.

I did not really understand why the network layer was being implemented. But, if it was implemented because of this vulnerability, I wonder if we should notify users that their private messages could possibly have been breached (by friends) between March 2018 and September 2018?

And of course, I would need to look into the security of lite clients, if this is the only kind of client that is effected by this issue.

An easy fix for me would to make it possible to disable private message indexing on the server.

Beyond this issue, it also concerns me that we're storing private messages in plain text from a security perspective. It'd be good to be able to easily turn this off.

looking at this again, the exposed query.read to friends is the problem and should have been raised when PM decryption was made transparent. I didn't know that was possible to begin with. As a stop gap we should maybe only listen to ssb-ws on localhost or rip it out all together.

Pubs running this shouldn't be that much of an issue, I think, since they don't have the key to access PMs not for them and don't get PMs frequently, at least in the uses I know of.

cc @dominic

looking at this again, the exposed query.read to friends is the problem and should have been raised when PM decryption was made transparent. I didn't know that was possible to begin with. As a stop gap we should maybe only listen to ssb-ws on localhost or rip it out all together.

Yes, I think either private messages should be stored encrypted or ssb-query shouldn't be exposed to friends over ssb-ws. It's kind of an either/or thing that wasn't really talked about when private indexing was implemented.

Pubs running this shouldn't be that much of an issue, I think, since they don't have the key to access PMs not for them and don't get PMs frequently, at least in the uses I know of.

Yah, I'll just turn the private message tabs off in the lite clients I'm running until a solution is discovered for this.

Thanks for taking the time to look into this @cryptix!

I want to give a brief update on the status of this vulnerability/bug.

Between @Christian Bundy @cryptix and I, we've determined this to have been an unintentional bug that was introduced when scuttlebot got private indexing, and fixed when the connections layer was introduced.

I think this means many of the folks out there who are using the latest scuttlebot are not effected, and should be able to consider their private messages secure. (Well, as secure as anyone should consider a highly experimental cryptography project that has not be audited by a neutral third party.)

I've taken the preventative step of turning off private messages in my publically served lite client at http://decent.evbogue.com/ , until I can figure out how to disable private message indexing and/or get it working well with the latest scuttlebot and the lite client.

If anyone else is using lite clients, you should consider doing the same -- if you're offering connections to peers who are not using the same public/private keypair as the server.

The only other team I know if who is using lite clients is @regular and @jfr, so ccing you guys to make sure you know about this bug.

Another update: I've merged a PR that removes these special privileges and released the result as ssb-ws@6.0.0. Please ping me if you have any trouble updating, as I'd love to get everyone running ssb-server@latest.

Also, I can confirm that ssb-ws does work with ssb-server since we have a test!

What is the current status of private messages in SSB? I am using the latest patchwork and sometimes patchbay. Should I be concerned about my private message content leaking? - @bobhaugen

Good question.

Another update: I've merged a PR that removes these special privileges and released the result as ssb-ws@6.0.0. Please ping me if you have any trouble updating, as I'd love to get everyone running ssb-server@latest. - @Christian Bundy

hey! I didn't see this post, because you did not mention me on it.

@dominic's statements on your issue are factually incorrect, the lite client is still being used by both my team and regular's team.

In addition, I find dominic's denial of my existence deeply troubling.

The issue with the lite client is that private message data is being stored in plain text.

I think security professionals would agree that storing private data in plain text is a security issue.

I'd love to use ssb-server@latest, but it also needs to support my lite client use case instead of pretending that it does not exist.

If the ssbc team cannot support the lite client, and turning off private message indexing, then I will create a fork of ssb that does.

@bobhaugen, only the private messages you can decrypt (i.e., messages meant for your eyes) are exposed that way, not everybutt's private messages.

Does that mean those are the only messages that are stored in plain text in my local ssb data?

Correct.

The ones I could read anyway?

Exactly.

And all other private messages are stored encrypted in my local data?

Correct.

The reason why it's completely unacceptable for lite clients is that everybody is able to read both their own and everybody else's private messages.

@ev does the problem of exposing private message data only appear in lite clients? - @bobhaugen

This should be true now in the latest versions of ssb-server.

However, between March and September 2018 it was possible to request private messages from friends over ssb-ws.

While it's unlikely that use this attack over ssb-ws, I think it's wise for the current ssbc to disclose that private messages could have been insecure during this time.

But @Christian Bundy your merge referenced in https://github.com/ssbc/ssb-ws/pull/15#issuecomment-469061078 fixes the vulnerability in the server but also kills lite clients?

Yes, one way to fix this vulnerability is to kill lite clients.

Another way would be to disable private message indexing. This is the solution that I'd prefer, if I'm to continue to use the latest ssb-server.

The reason why it's completely unacceptable for lite clients is that everybody is able to read both their own and everybody else's private messages. - @kas

Correct.

How does that work? (That lite clients are able to read all private messages?) - @bobhaugen

Hmm, not quite true. The lite client can't read everyone's private messages.

The issue is that with ssb-ws you could websocket into another sbot and request their indexes, which contain decrypted private messages.

So with a lite client from a pub, you accidentally end up sharing private messages sent to the pub with the public web.

Here's an example exploit, just to be clear:

Alice and Rob are on the same local network, or otherwise know how to directly contact each other's sbot.

Alice requests Rob's query indexes over ssb-ws.

Rob's ssb-query sends his indexes over muxrpc to Alice.

Alice records the ssb-query index to her computer.

Later, when Rob isn't around, she reads all of Rob's private messages that he'd decrypted on his own machine with his private key.