A process for our specs

#SSB has a bunch of specs nowadays, and we're (#batts team at least) making more. To mention a few: metafeeds spec, rooms 2.0 spec, broker auth.

Problem

Our specs are scattered, hard to find them all, hard to refer to them succinctly, and unclear who owns them, how they evolve, etc.

Proposal

Years ago we actually started organizing this, thanks to @cft's effort to kickstart ssb-spec-drafts. We need to revive that, update it with the N specs we have in the wild, and improve the process, clarify the governance, etc.

For some inspiration:

• XMPP's specs
• Zcash's specs

Help!

I can't just take this extra task on my shoulders, I got too much going on. We need help to:

• Revive ssb-spec-drafts, perhaps renaming it as (my personal preference) "SIPs" (Scuttlebutt Improvement Proposals)
• Collect all the spec repos we have in the ssbc org and put them in that one place
• Index all the specs by their status and which apps implement them
• Define process and governance
  • Who "approves" a spec? Based on what criteria?
  • Do specs have to be global consensus? Can we instead just let each app list all the SIPs that they implement?

cc @arj @nonlinear @mix ² @SoapDog (Macbook Air M1) ² @Matt Lorentz (planetary) ²

This is really important. Thanks for bringing it up again. Gardening the specs can be a lot of work. Perhaps we should pay out of one of the OC’s a monthly grant to someone to take it on?

Here's a prototype of what spatially arranged specs could look like

https://github.com/mixmix/scuttle-specs - note all the nodes are clickable, and take you to the spec (IF it exists!)

Sorry folks, just for the sake of planting my feet firm on the ground. I suspect there are two convos happening, or two different convos should happen.

The original thread felt to me more like an effort to organise and make the specs available somewhere, which is what I'm trying to do with #ssb-documentation, but the thread pivots into governance and stakeholder approval which is a totally different conversation. Both are needed.

@mix.desktop I'm totally gonna vendor that spec repo into the #ssb-documentation repo, ok?

The original thread felt to me more like an effort to organise and make the specs available somewhere, which is what I'm trying to do with #ssb-documentation

@SoapDog I assume your documentation is linking to those specs, right? What I meant with organizing is that we should:

• Move the markdown (or whatever format) from those separate repos into one common repo that contains all the specs
• Make that monorepo the "official collection of specs"

The centralization of specs means that they will also require governance. So I don't see two discussions.

@andrestaltz,

I'm using git submodules to add those repos into the ssb-documentation repo. This way, each team working on a spec can keep using and updating their repo as needed and we can easily fetch them and use them from the documentation repo.

I decided upon submodules — which I am experimenting with this very second — because it is very low friction. It doesn't require anyone to change how they work and it is easier to maintain than manually copying stuff across repos. Also, it has less impact on governance.

I just posted another message detailing how I got the diagram that mix built to work on the documentation repo.

Just showing the specs integrated with the current SSB docs repo.

I'm not a web designer. I actually need help with a better template, but this is good enough for my current use.

This shows the specs working and also the integration of their structure into the table of contents. Even though it is converting every Markdown inside those repos, it is only using the items from the README to assemble that ToC. If I let it use all Markdown files in all repos to build the ToC it becomes so long that it is unusable.

All the specs have been added to the repo as git submodules. They appear as normal folders.

The configuration.toml contains references to them:

title = "Specifications"
toc_depth = 1

files = [
"README.md",
"bendy-butt-spec",
"envelope-spec",
"private-group-spec",
"ssb-bfe-spec",
"ssb-buttwoo-spec",
"ssb-meta-feed-group-spec",
"ssb-meta-feeds-spec",
"ssb-uri-spec"
]

Teams can keep working on their repos just like they do now. I can easily update the documentation using git submodule commands. There is no need for people to manually send PRs or contribute to this repo first. It doesn't add friction to our current governance model.

The same approach can be used to add content from the developers treasure trove and from any other repo we fancy.

wow this is a nice surprise to wake up to @SoapDog (Go) / @SoapDog (Macbook Air M1) <3

Heads up that diagram was a prototype and could do with some review!

Spec calendar

I realised in the night that it would be highly beneficial to have a spec'ing calendar. I notice lots of people are like "did gatherings 2 happen, did I miss it?". A calendar where people can spread out the processes over the next months clearly would enable people to know:

• what's coming up
• how / when / where to engage
• minimise attention burn (ideally only have 1 spec under dev at a time)

This doesn't need to be centralised, but having a centralised notice-board which a decentralised crowd can coordinate around would be swell.

@andrestaltz📱,

Maybe updating specs is a bug, not a feature.

I see it quite differently. Most specs I know have been updated multiple time. Not only needs change over time, but the problem domain and the spec capability of handling said problem domain is better understood as the specs gets used.

All the web specs have multiple versions. Most network protocol specs have multiple versions as well. Even printed books get erratas. Evolving specs is a sign that they are being used, no one gets it right the first time.

And maybe specs should be a point of consensus (thus centralization), not decentralization (repos distributed across many owners).

Well, consensus is not our strong suit here. Four years and we're still talking about URLs, and that was supposed to be an easy spec for us to reach consensus.

I'd settle for a clear workflow and resources to follow so that people understand which clients implement what vision of the protocol.

This is why I’m saying that moving repos and governance is one thing.

As you said, I'm being proactive developing a way for people to discover and consume documentation without forcing them into new workflows or requiring too much consensus.

The good thing is that if we ever get around settling on a governance model and some consensus regarding specs, the current SSB documentation repo won't require any change to keep working and benefiting the ecosystem. It will simply become easier.