You are reading content from Scuttlebutt
@andrestaltz %/qbAfJ5ZKKz57a/BWIU2YxGtQNNdeLd+DblQ1aj2X1I=.sha256


I tend to make the strategy for Manyverse clear (see roadmap), but my strategy for SSB is slightly different. Instead of boring the reader by repeating my thoughts on Onboarding, Sustainability, and Safety, I'll try to share my vision for SSB that is necessarily not Manyverse.

Of course, there is some overlap. Lots of goals that are part of Manyverse will directly benefit the SSB ecosystem. For example, in the past: ssb-conn, rooms, ssb-blobs-purge, ssb-keys-mnemonic. And examples for the future: onboarding via broker auth or something similar, etc.

So let me talk about the non-overlapping goals.

First, I really would like to see a hundred apps thrive on SSB. I don't want Manyverse to suck up all the possible use cases. I think diversity is rich and sometimes you need to break away from the constraints of "a normal social app" such as Manyverse, to build wild things. I myself have a handful of ideas I'd like to build. I can't wait to see what else people build.

So a hundred apps. 100 is just a symbolic number, but it also means that a lot of things have to happen before we can see such thing happening. Developer experience has to improve, etc, but I think biggest of all is that app-to-app interop has to be seemless, has to "always work", has to never become a source of headaches, and has to be well specified and documented.

One hint towards that direction is my opinion that apps should not share the same database. The common flume database used to be the primary point of interop for apps like Patchbay, Patchwork, Oasis, etc. This way of interop has been tough already with just a handful of apps, they constantly stepped on each others toes indexes, imagine then with 100 apps. Removing the common database from the picture means there has to be something else, some other convention for sharing data. I believe that convention should be something I call "A2A" (app-to-app), which is just a fancy way of saying that apps should connect to each other just like normal peers connect to each other, and then there should be some authorization flow (this is what broker auth is) to allow app A to get special data from app B (such as the social graph). It doesn't have to be broker auth necessarily, it can also be shs2 so to allow LAN A2A.

Unfortunately, we can't just build A2A today. Well, not without creating a bunch of headaches. So I propose we solve those headaches first, and then it'll enable building A2A in a proper manner, allowing apps to be created easily and without freaking out developers.

To repeat a blob I shared before, notice "Cross-app bootstrapping":

top-bottom graph with: database compactions, index feeds in production, private groups, brokerAuth invites or shs2, fusion identity, and cross-app bootstrapping

The dream of several apps on SSB was revived during our #ssb-ngi-pointer project when we realized #metafeeds would be great not only for partial replication, but also for app-dedicated feeds. This way, when you play chess with friends, you can replicate only chess feeds, not all the other stuff. If you are not interested in chess, then you wouldn't replicate it, because you wouldn't even have a chess app.

But you should still have a way of tying all these feeds together. You would have "subfeeds" (created hierarchically) for apps on the same computer, such that they all point to a parent metafeed. But you could also have "side feeds", i.e. #same-as for linking desktop feed with mobile feed, etc.

#metafeeds is a way of linking feeds hierarchically, and #fusion-identity is a way of linking feeds horizontally, in other words #same-as. These two aren't ready for production yet, and that's something that needs to be worked on before we get to "cross-app bootstrapping".

And before we can put metafeeds into production, we need database compaction in place, for reasons I already shared before.

So my strategy for this year is summarized in the graph chart I shared above.

Hope this resonates!

@Matt Lorentz %VIacrc2FzSqUEOQi+1BqebGHfwleZ3Jkb3T5zdKkjbM=.sha256

@andrestaltz metafeeds and fusion identity are so exciting and will really be revolutionary for scuttlebutt! I'm so glad you and the ngi folks are driving this forward.

I know iOS is probably going to be the most difficult place to do A2A with the strict sandboxing rules Apple enforces. I'm happy to lend my iOS skills there if you can ever use them.

User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
@andrestaltz %hRnKqCwnTyvLbIhljJSCXZXaYx332NOmxZi3ApbFkF0=.sha256

100 scuttlebutt apps. Can you tell us why that is a goal? What do you hope it will achieve/ unlock?

@Mix It would help SSB grow in a healthy manner. SSB could grow as a product of Manyverse and/or Planetary growing, but the result would be a bottleneck, a sort of "centralized" situation where all responsibility falls on the shoulders of few. And since responsibility=power, it would also mean that power lands on the hands of few. In other words, I want SSB to grow, but in a decentralized manner. Decentralization means freedom to choose, freedom to migrate/swap, etc.

To be fair, I do want Manyverse to grow, but I don't want it to be the only (or one of the few) SSB app growing, I just want it to be the "SSB beginner's app".

If you look at the simplicity of creating web apps, that aspect of the web is healthy because there is growth and decentralization. On the other hand, the complexity of creating a web browser means that this aspect of the web had unhealthy growth because so many things are dictated by Chrome (Google) which hurts freedom of choice.

why db compaction before a2a? I am guessing you worry that a proliferation of apps means our dbs will fill up fast, but I don't feel that

I absolutely believe it would fill the DBs fast. I think you need some numerical convincing of this fact, because it's not something to "feel", it's something to measure and to prepare for before it starts happening to everyone.

For instance, on onboarding, we already experience OOM crashes due to lots of different (large) indexes being built simultaneously. Dominic got that with Manyverse on desktop, I'm curious how much RAM he has. My computer has 16GB and I've never gotten an OOM crash on Manyverse during onboarding. So let's assume he has 8GB of RAM, and that it's a fairly usual computer. That would be consistent with other reports of Manyverse mobile crashing on phones with 4GB or 8GB of RAM.

Enabling partial replication for common content (such as votes, posts, contacts, abouts) is paradoxically going to use up more storage, until we get sliced replication. And introducing sliced replication only makes sense if we also have deletions (in a ring buffer style) and db compaction. So assume you don't have deletion and compaction in any manner. vote / post / contact / about messages are roughly 75% of all messages according to this study by arj. "Partial replication" of those in practice means enabling index feeds. Index feeds means that for each vote/post/contact/about message there would be 1 index message published. Index messages are small, but they are still conventional SSB messages, probably the same size as a vote message normally is. So take the existing 1GB log and add +75% to it and suddenly the log is now 1.75GB large. That would cause it to get a OOM crashes more frequently. Add to that some new messages from new apps, and it would get a lot larger. I mean, I'm thinking 100 apps, not just 3 or 4 new apps. Some of those apps will be naively publishing messages (and rightfully so, because naive is often the most approachable for beginners and people just experimenting/getting-started with SSB, which I want to see more often). It only takes one naive app to publish a ton of content for your database to get full, and I'm only talking about OOM crashes, not yet about the hard limit of 4.3 GB.

I don't want to encourage an ecosystem of apps to build on a foundation that is going to crash soon. It would stain our reputation as a non-foundation for apps.

User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
@andrestaltz %mwRkUu4UQrQ7CqNxlJGltO2TC7aRRU4l5b8WL9Z+edA=.sha256

Can I read more about this idea?

@Mix Quick summary is that you would have this hierarchy of feeds:

  • Root metafeed
    • Main feed (this is the "classic" SSB feed, e.g. the one I'm using now)
    • Indexes metafeed
      • Index feed for post
        • Msgs in this feed have cypherlinks to post msgs on "main"
      • Index feed for vote
        • Msgs in this feed have cypherlinks to vote msgs on "main"
      • Index feed for contact
        • Msgs in this feed have cypherlinks to contact msgs on "main"
      • Index feed for about
        • Msgs in this feed have cypherlinks to about msgs on "main"

To read more about it:

User has not chosen to be hosted publicly
@andrestaltz %qD1jrLGii2/7mGIjXpTaPKPtXwpzbUGDTSVxq1v0P7s=.sha256

@Hamiller to clarify, the end-user would be unaware of index feeds. They would not be able to choose which feed on which to post a msg. It would all work in the background. The only purpose of index feeds is to allow friends to say "give me all contact messages for Alice, but not the other messages", so that they can replicate less stuff, e.g. leaving out votes which take a lot of space. This means in practice you could browse someone's (say a stranger's) profile but only to see their name, picture, biography text, and followers, not the actual posts. Or maybe you could choose to see the last 100 posts, but not see their votes. That kind of thing. The purpose is simply to allow choosing which content types to replicate.

User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
Join Scuttlebutt now