Implement SSB in C

Proposal for June 2018 SSBC Grants

I would like to implement a minimum-viable scuttlebot-compatible SSB pub server in C.

Goals:

• Run comfortably on small OpenWRT devices and cheap VPSs.
• Accept peer connections and schedule outgoing ones.
• Gossip blobs and messages/feeds.
• Validate messages

Previous work: sbotc, dill-boxs

I love it. Would be awesome to run a pub directly on a router.

hell yes! also this would benefit from the separate process plugins proposal that @keks @cryptix and I have been discussing. (and that we need to publish...okay I'm gonna write something up)

@cel also did a great job on sbotc I was quite impressed that he managed to implement everything needed for a sbot client in just one program. It's also so small that people have used it from other languages, say, to shell out from python.

@corlock
A full sbot implementation I would expect to have all the API of scuttlebot, including the secure-scuttlebutt module and the default plugins. I think it would be a huge effort to build and maintain that. The full scuttlebot API includes methods and indexes which are useful for building applications but are not all needed for replication - and are also subject to change. The methods used for replication, i.e. scuttlebot's public API, are a subset of the full API, enough to participate in the network as a full peer, slower to change, and more well-understood they are or will be, I think, implemented by multiple people making SSB implementations.

A ssb client implementation in C could mean different things. sbotc is already a ssb client implementation in C - for the specific use of running sbot RPC calls as processes. A general-purpose client implementation in C might be a library for programs to use. Or it could be a SSB UI written in C - a set of commands, a TUI or GUI. Possibly a library could be factored out of the server implementation. But this would not necessarily be needed to make a standalone server which is primarily what this proposal is for.

Implementing in C allows the server to run on many systems. It allows for fine-grained use of system resources. Possibly Rust or Go could do that too. But my offer is to do it in C.

Every programming language/environment/paradigm is an opportunity to implement SSB, to realize
the benefits particular to that language/environment/paradigm and its community, and to increase the knowledge, experience, and resilience of the network.

@Nico
In general I think messages will arrive at a low enough rate that the CPU should be able to handle verifying hashes and signatures as messages arrive. For a simple full peer, no decryption attempts of messages will be necessary. When there is a high rate of messages, such as during initial sync, the program can read them slower from the peers sending them, creating socket backpressure (and/or muxrpc backpressure if muxrpc credits are implemented).
Storage would be an issue: a small router would probably need external storage for the messages and blobs. Unless the implementation was going to not be a full peer but either

• primarily proxy connections between peers, or route p2p signalling handshakes.
• not store entire feeds, but only the sequence/latest and/or computed state of them
• store only a small fixed set of feeds (e.g. only its own feed - antenna pub)
• not store blobs. or store up to a limited amount of blobs.

@dominic
Yes, either of those plugin proposals would work well with this proposal.

I wanna emphasize that @cel is doing the right thing here just targeting replication. There are lots of apis exposed that are not used (but kept there because some test uses them). hmm, though I think the apis that patchwork uses largely come from plugins so maybe if it was possbile to run those as separate processes, then it would be easier to support patchwork (although other UIs might be different)

I could be ignorant but I think some of this overlaps with @piet's ssb-mobile-quest?
That's not a bad thing, I think rather it's an opportunity to share learning and further strengthen things

@CustomDesigned any particular area you are interested in?

I am working on getting the server to communciate with its peers: %ICu6NQw....

One thing that could be implemented in parallel would be message validation and hashing (and signing).

@Nico
git clone ssb://%4w0BO1KXW+6zBSJlQx1YViQeknl16anzhesmwKGRh2g=.sha256
It is still early stages though - not very useful yet.
Related: %hQpsBXW...

@w this project is stalled.

@makeworld I would first like to do a voice call with any SSB community members who would like to talk with me about this. Please let me know if you are interested in that.