You are reading content from Scuttlebutt
@luandro %tz/6ms3u5cN5y3pKtRhtCkC4eXp7ASnhj6KU1tnA6bA=.sha256

Meshtastic integration with Secure Scuttlebutt

Reposted from https://meshtastic.discourse.group/t/meshtastic-secure-scuttlebutt

Secure Scuttlebutt (SSB) is a decentralized data platform, specifically made for offline-first social networks. There are many similar decentralized platforms out there, famous ones this days seem to be Mastodon and Matrix. But for my use-cases, communities that have no connectivity, they both lack true p2p, relying on servers, which disable device-as-infraestructure.

There are spaces where federated and p2p have been converging, Mastodon example and Matrix example. And as convergences happen they start facing similar issues, like identity spread around multiple devices and device storage for example.

There's lots that makes Scuttlebutt special, specially the community and end-goals behind the protocol. But technically, it's been growing for the offline/off-grid first perspective. So it's advanced a lot on how to enable security and at the same time discovery for completely p2p gossip networks that don't rely on DHTs.

The reasoning behind integrating SSB and Meshtastic is that they both provide the cheapest transports possible. First SSB turns every device into a node which communicates within Bluetooth or Wi-Fi range and can carry messages sneakernet-style. LoRa is the most accessible long range transport out there, so it's naturally the next option for extending a network, say to another community for example.

Next steps would include a Wi-Fi long range link, HF, satellite or cable. But all of those are in another order of magnitude of accessibility (service-provider availability, licensing and cost) and usually open up to the Internet, which is not desired by every community.

Challenges

SSB relies heavily on cryptography to be secure. Hashes are long pieces of data that are terrible to carry over a low-bandwidth transport such as LoRa. A simple "like" in SSB looks like this:

{
 "key": "%a8acTq/Bss5G1j7JsggyHibLJ+PIM/1XvhifOLEXqUA=.sha256",
 "value": {
 "previous": "%z0IQ/vul09vRhnk+nvpVGrzHE6K7UD99h9NdL2gr+8s=.sha256",
 "author": "@2RNGJafZtMHzd76gyvqH6EJiK7kBnXeak5mBWzoO/iU=.ed25519",
 "sequence": 377,
 "timestamp": 1518257352380,
 "hash": "sha256",
 "content": {
 "type": "vote",
 "channel": null,
 "vote": {
 "link": "%ehhSqb2l/GbqhdWhvrHs0LkfTjA2LB4krK/kdY/eCqg=.sha256",
 "value": 1,
 "expression": "Like"
 }
 },
 "signature": "yrXKc7C5riUVXB6ckaLZ7zzvn63G6AdzIEPM1esuu3EDS2whDr5DL2N8ILPGoa4pzp9cMd4O6kJEJEkr1/IPB==.sig.ed25519"
 },
 "timestamp": 1580754824704
}

That's 543 bytes for a Like message×, about *400 bytes just for the envelope. But a message can easily be well over 1Kb. All this in JSON format of course.

Is this even possible?

We obviously don't want to replicate the whole SSB network over LoRa, that's insane. But it makes sense to replicate messages of key types. For example, we'd want to replicate identities (public-key, identity-fusion, name, location) over Lora, and maybe public and private messages of a type "Emergency".

The networks we're thinking about are composed of neighboring communities. So we can expect and play with certain amounts of trust (less security concerns).

ssb.proto would look something like this:

message SsbMessage {
 message Value {
 string previous = 1;
 string author = 2;
 uint32 sequence = 3;
 uint64 timestamp = 4;
 string hash = 5;
 string content = 6;
 string signature = 7;
 }

 string key = 1;
 Value = 2;
 uint64 timestamp = 3;
}

We could use distributed hash tables over LoRa and a shared secret (maybe the Meshtastic channel name) to hash the hashes into smaller-sized hashes that are re-constructed after each send/receive. As our chances of collision will be very small in a such a small network. But don't even know if it's worth it.

We could also automatically break the messages into smaller chunks, sending first headers then broken up chucks of content.

But then the biggest issue is on SSB's side. It keeps an append-only log for each feed (key pair), where every message refers to the last one, so that logs are temper-proof. Receiving a special message from LoRa without having previous messages to check will break its security. But it's probably a worth-while trade-off. Is this what #ssb-partial-replication does @arj?

Conclusion

The end experience we're aiming at is for complex, multi-user, multi-threaded interactions that SSB offers, while using LoRa as transport in remote locations. I believe this and voice-over-LoRa are key features that would perfectly fit within the ecosystem of applications that can satisfy needs of networks of offline communities.

Thoughts?

User has not chosen to be hosted publicly
@Anders %9uXb7vFT0ISf+LLAWZ964uh1Nim3gLsMaocR/013RLc=.sha256

@Luandro Pàtwy first to answer your question. SSB has always had the concept of OOO, meaning getting a message out of order. The message is valid in itself given that it is signed by the author. Interestingly if the messages themselves includes references to other messages that belong to the same subject (tangle), then you don't really need all the previous messages. This is what tangle sync is about. That is one form of partial replication. And if the messages are completely independent then getting them OOO style is completely fine.

Regarding message size I would probably look into bamboo and bamboo-rs that should work much better on lower powered devices. I believe @piet knows more about this.

I also remember that @cft had a fascinating talk on a related topic at the Basel meetup where one of his points if I remember correctly was that because the messages themselves include integrity, you don't really need it all the other layers of the stack to make sure that message are not corrupt, arrive in the right order etc. Can't find the slides now, but I'm sure he has something interesting to add here as well.

User has not chosen to be hosted publicly
@mirsal %HPjW8FSyE5jQiK2eUMQ1rIi/Z9x9AV2qywsrPzvFzTg=.sha256

That reminds me of experiments by matrix devs with ultra-low-bandwidth transports, there were good ideas in there, maybe some of them are relevant to SSB on LoRa: https://matrix.org/blog/2019/03/12/breaking-the-100-bps-barrier-with-matrix-meshsim-coap-proxy

User has chosen not to be hosted publicly
User has chosen not to be hosted publicly
@Cinorid 💻 %SN4bE+mEWlJ3vYGCT7sVWCkaZANk/PMVYGxC6e6f8A4=.sha256

@Luandro Pàtwy I think if the data is sent in a binary way (similar to bipf), we can significantly reduce the size of data sent without changing the structure of the packets.

@luandro %2aeYV7o65KEZrI9Gidn3nhEuC7SjzFtMbN7Vsx4SzBI=.sha256

@mirsal This is gold! They went far lengths to achieve 100 byte messages. Could we achieve something similar for SSB?

Just watched the amazing presentation on SSB DB2 by @andrestaltz, donno how I had missed that.

@arj thanks for explaining, so ordering is not a demand? Really great to revisit Bamboo, I feel I better understand it's concepts and what it's trying to achieve. But it seems yall did an amazing job with #ssb-db2 and, as @Cinorid 💻 mentioned, bipf is pretty smart.

just transfer over LoRa the first 4 characters of the msg cypherlink

Prefix indexes 😉 Genius!

Alice and Bob would not have the same metadata (msg key, signature, creation, timestamp, full author key) replicated, but at least the would have the same content replicated.

But such a message wouldn't be validated by ssb-validate, how would I send or replicate it? The example you gave in the presentation had a 27 bytes example object, what would you say the typical size for a ssb message such as this post I'm writing now, in bipf?

Meshtastic already has MQQT implemented on their side. Looking into it it seems it has "an overhead of 9 bytes on top of the payload", which I don't fully understand but seems pretty damn small.

The trick seems to be on how to compress/decompress ssb messages using prefix indexes and keep only the necessary for it to still be valid for ssb to replicate.

@luandro %Wbo0q3h/6c2kF63RvKe0ZI+JetF6m8PKMJ4/dTgOqnw=.sha256

Using bipf.encodingLength(msg) on the previous message got me 2876 bytes which is a lot. But that message it rather complex, with lots of mentions, branching and reply information.

The least I'd need for a message is author, sequence(?), key, root and text. Something like this:

{
  key: "2UeY",
  sequence: 8094,
  author: "2RNG",
  content: {
      text: 'Hello',
      root: "tz/6"
  }
}

This is just 68 bytes 🎉 not bad! A 140 character message is around 200 bytes in bipf encoding, not bad at all.

Using prefix indexes for ids and threads. It seems there's a 258 byte limit, so we're there.

Could such a message be added and replicated by clients such as Manyverse?

@luandro %pGdE4ckDBiBBqe2c6jbsn3GZxxw0k1yI+lLqw/P4fKM=.sha256

From SSB protocol guide:

Message format

 {
  "previous": "%XphMUkWQtomKjXQvFGfsGYpt69sgEY7Y4Vou9cEuJho=.sha256",
  "author": "@FCX/tsDLpubCPKKfIrw4gc+SQkHcaD17s7GI6i/ziWY=.ed25519",
  "sequence": 2,
  "timestamp": 1514517078157,
  "hash": "sha256",
  "content": {
    "type": "post",
    "text": "Second post!"
  }
}
  • previous: as Arj noted, this is not strictly necessary, but how to we pass ssb-validation with this?
  • author: as Andre suggested, we can use just the 4 initial chars as prefix, and re-construct them on the other side (special id LoRa messages containing the full pub key and name would have been sent before-hand)
  • sequence: is necessary for other nodes to identify it's a message they don't have from this feed, correct?
  • timestamp: we could use LoRa device's timestamp for this
  • hash: we can trust all nodes to have the same
  • content: this is the heavy part, which we'll enforce on the client-side to use 140 characters

Signature ⚠️

All messages in a feed are signed by that feed’s long-term secret key.

This is tricky, and can't be re-constructed on the other side without leaking the private-key and it's an extra 117 bytes, which makes it impractical, requiring at least two packets to be sent ❗

Message key

A message ID is a hash of the message including signature.

This can be reconstructed on the other side as long as we have the signature.

Use case

  • LB (local offline network browser-user)
  • LM (local Manyverse user)
  • LP (local Pub)
  • IB (Internet browser-user)
  • IP (Internet Pub)

LB sends a public identification message to the LoRa mesh

LB --> name, full_pub_key --> SSB (about) --> compression --> LoRa mesh -->

All nodes eventually receive the identification message. Clients (just browser for now) pull the message from the LoRa node and reconstructs the about message (using trusted or already known secrets, such as Meshtastic channel), adds it to their db, and replicates it further using any means available (Bluetooth, Intranet, Internet).

LB sends a private message to IB

Since we don't know where the message receiver is at any given time, LB sends a private message to the whole LoRa mesh:

LB ----> boxed_msg ---> SSB ----> compression --> LoRa mesh ----> allNodes

       LB-11, LB-12 <--- SSB <------ decompression <----- LoRa node1 <-----

        LB-21 <--- SSB <------ decompression <----- LoRa node2 <-----

(Internet) <--- LM-31 <-- LP-31 <-- SSB <-- decompression <-- LoRa node3 <--
   |
   |
   |---> IP ---> IB ---> unbox

All LoRa nodes eventually receives the message and stores it.

LP-31 pulls messages from node3 decompresses the post message, adds to it's ssb-db, and replicates it.

LM-31 replicates with LP-31 through the local network.

LM-31 goes to town and replicates with IP

IB connects to IP and eventually receives the private message.


Signatures seem to be the byte-size bottleneck so far for SSB integration.

@luandro %H03FlCycI4QXVF8ieSvRD+RDVwqT+S7bnBhaelSu1sY=.sha256

And if I understood correctly private message will have at least an additional 73 bytes, 49 bytes for each secret_box and 24 bytes for the nonce. Reducing private messages to less then 140 characters, but possible...

User has chosen not to be hosted publicly
@luandro %xXjhoT91OZgKfPQ9NEmjJLwmPBmcv9MdbtKPY6EPelE=.sha256

@emile I haven't given that much thought before. In the first experiment I think we were using Medium range (but fast) - 5.469 kbps (Default), but I'm not 100% sure (poor documentation on my part).

There were no tools for debuging the mesh, by today with Meshtastic 1.2, theres the plugin system, and the range test plugin which makes it a lot easier to test. With the JS library creating better interfaces to debug has become easier. Because the nodes were placed in such remote places, the only way I had to test was by sending messages using the phone. So I got hardly any information on the link quality/stability.

But I should probably write a script that changes settings based on synchronized cycles of time, for testing different settings. But this could even be a way to gain bandwidth and avoid collisions 🤔

User has chosen not to be hosted publicly
@luandro %W384Dec/0PrEuN0VuR6xTd3ObrtNSVFJ97kSoB/qAXE=.sha256

@emile here's the Meshtastic channel settings. I wouldn't mind going with illegal settings for tx_power for example. It provides quite a lot of api to play with.

I'll readup on Aloha, starting from the newest publication, because it has the coolest name: THE-ALOHANET-—-SURFING-FOR-WIRELESS-DATA.pdf

User has not chosen to be hosted publicly
@luandro %9dXJSotpDXpH6AxfcUxxxV2fQ7AOzO34Upn5JTeyV84=.sha256

First working prototype

A couple of months ago, with @andrestaltz's and @sashaw's help, I've managed to modify the still experimental meshtastic-api to run a pub/bot which publishes #meshtastic messages to SSB 🎉 Here's the repo.

Still very very crude, but functional. Really excited about the possibilities for this, such as:

  • using a #community-server running the pub to store messages
  • not needing a Bluetooth connection between phone and radio
  • using #manyverse to communicate over LoRa
  • exchange LoRa messages thru the Internet by having the pub connected to a #ssbroom

Next steps would be to reduce message sizes as much as possible, split them up, and send them over LoRa in a way that we can add them to the SSB db so that the author themselves appear as publishers. But there's still a long way to go to get there.

Join Scuttlebutt now