Grant Proposal: An SSB Client Library in Rust

What:

Scuttlebutt is a distributed database with network protocols for communication between peers. All clients (e.g. Patchwork, Patchbay etc.) use the same set of protocols to talk to the local database. The only full implementation of these protocols is written in javascript. Consequently, all current clients are forced to use that language.

With this grant, I'd like to implement the protocol suite in Rust and provide a client library corresponding to the js module ssb-client.

Why duplicate code in another language?

The current lack of diversity in protocol implementations mandates the use of javascript, even though the core design of ssb has nothing to do with javascript in particular. To become a widely adopted protocol, there need to be implementations in more than just one language. There are also some problems specific to javascript:

• Graphical clients have to rely on web technologies. This means either running in a browser, or using electron. Both of these choices have considerable drawbacks compared to native GUI frameworks.
• People conflate ssb and web technologies. Even if the protocol itself remains language agnostic, client conventions develop based on the available clients. And if all clients are written in js, it becomes tempting to adopt web technology features into clients (e.g. html or svg rendering, embedded js, mathml, etc.). This would make it tremendously difficult to write clients in other languages. The earlier we achieve language diversity across clients, the lower the chance of this happening becomes.
• Different people prefer different styles of programming languages. Programming language design involves a lot of trade-offs. Different use-cases might require a language with different design choices. More things will be built on ssb if you are not forced to use one particular language with its specific set of trade-offs.

Why rust?

Rust is a compiled, statically typed language without garbage collection. Unlike javascript, it emphasizes efficiency. Compared to js, the rust modules should both run faster and consume (much) less memory.

Since rust compiles to native code, it is possible to provide bindings to the rust implementation for other languages. Providing bindings to a client-library takes less effort than reimplementing (and learning) the whole protocol stack. So an implementation in rust would significantly reduce the effort to be able to write ssb clients in your favorite language X.

The major native GUI libraries can be used from rust. Clients written with these libraries don't have the large overhead of a web browser.

Static typing and the excellent documentation comments make it easy to use the library. This can also provide a good starting point for other people to learn about the protocols. Having standardized, compiler-verified documentation about all the inputs and outputs of all public functions is just something different than the free-form documentation on scuttlebot.io.

Other Info

Implementing the whole protocol suite not only enables a client library, it's also the first step toward a server implementation. A reimplementation of the database and replication logic is out of scope of this grant request. But in the long term, it would be extremely valuable to have an efficient and compiled sbot implementation.

This proposal nicely complements @vtduncan's grant proposal for technical documentation. We'd both be learning about the protocol suite at the same time, and I don't think there are better ways for providing feedback on the docs than by actually trying to implement things based on them. A rust implementation might also be a good reason for the benchmark proposal to consider language-independent benchmark suites.

I already implemented the crypto-layer of the protocols in rust. I think these are the most difficult parts of the whole undertaking, which makes me confident that I'll be able to get this thing done. These implementations are released under the GPL, and unless there are strong objections, I'd like to copyleft all the rust modules.

I would not work on this full-time, instead I'd stretch the grant over two months. I'm going to university, so I can't code full-time as well.

#rust #ssb-implementations

that's really good to hear! i'm definitely interesting in using your lib in the future. i'm building a dex operating system using rust and elixir (and a few other things). right now i'm focusing on the ui libs, but in the future i want to tackle building an sbot implementation in elixir (with rust speedups if necessary). a rust client library would definitely be something i'd be interested in contributing to when i get to that part of the stack.

I think this is a great idea and appropiately scoped!

actually i take it back, rust-sbot would be perfect.