You are reading content from Scuttlebutt
@andrestaltz %6hdfDTtpOqb7BZYWvpyu9isRsBdmIgn2pziIQKBYpr4=.sha256

The days of ~/.ssb are counted

Manyverse Desktop is going to be different from other desktop SSB apps because it won't use the .ssb folder in your user home folder. I've been thinking about this problem (i.e. how should apps on the same device coordinate around your SSB data?) for years and I think one folder per app is the way forwards, in my opinion. Obviously many other bright minds have thought about this problem, such as @Luandro when working on his SSB apps on Android, but also the folks behind #scuttleshell, as well as @Mix and the #ssb-ngi-pointer team recently. This post is for explaining the reasoning behind one-folder-per-app, and I'm also going to put up a FAQ later on the Manyverse website about issues surrounding ~/.ssb.

Sharing the folder give us headaches

When apps share the same folder and database, they can easily mess up each other's functionality. For instance, update Patchwork and it may rebuild one specific database index which Oasis was using, and then Oasis is broken. Or vice-versa. Often people have avoided those headaches by using only one app regularly. Or they have to delete all indexes before taking the other app into use, to force indexes to be rebuilt.

Other solutions like a "hub app" or Scuttleshell share similar kinds of consensus-based headaches.

Flume versus ssb-db2

One hard reason that forces us to do something different in Manyverse is that Patchwork writes to flume/log.offset while Manyverse uses ssb-db2 which is incompatible with flume and writes to db2/log.bipf. These two logs cannot be written at the same time, otherwise you risk forking your feed. There can only be one source of truth, either one of those logs has to be the main one. Also, streaming messages from one to the other is worse for performance.

Easier to develop an app

Beyond just Manyverse, I want to see an ecosystem of hundreds of SSB apps, and it's easy to create an app when the app has 100% control over its own database. Optimizing for developer productivity and simplicity is a good bet for the ecosystem of SSB apps.

Co-locating with off-log databases

Say you need to store some data outside of the SSB log, like "unread state" or simple things like your choice of color theme in the app. Often these are stored in things like (or resembling) localStorage, and that's already true for Patchwork, it has some app data stored in ~/.config/Patchwork, not ~/.ssb/Patchwork. So it would be convenient that if you delete ~/.config/Patchwork (say, during uninstalling) it would also delete the associated SSB data for Patchwork. If we colocate the SSB log with the SSB app data, this is simple. And also, it's more easily backed up or transferred from one computer to another, because it's just one folder you need to worry about.

Some operating systems demand it

You can't have an equivalent to ~/.ssb on iOS (or at least it's very difficult), and Android just recently introduced a similar constraint, and this left me scratching my head for months, how to work around it.

If we follow the one-folder-per-app mantra, it should be supported in all operating systems, so it's a good and simple strategy regardless what platform you're developing for.

Metafeeds and fusion identity will help us

These new subprotocols will enable us to properly tie together data from different apps. The solution for "I want my account on all apps" isn't to have literally the same feed ID on all of them, but instead something more flexible like same-as (Fusion Identity) and subfeeds under a root metafeed. This way, each app can choose its own feed format (this is good for the future if/when we want to update feed formats!), without messing with other apps.

Downsides that can be worked around

One problem may be duplicate data. Say you have both Manyverse and Patchcool installed and both of them replicate data from other peers in their own logs. There might be a bunch of duplicate data taking up space on your computer. That's a problem, but there are solutions that could be developed.

In the future when we have partial replication (reminder, partial replication is 90% implemented already, a proof of concept already works), each app can replicate only the data it really wants, for rendering on the UI. So an SSB chess game would only replicate chess feeds, while Manyverse would not.

Another storage-related problem is with blobs, they could also be duplicated. But there are also case-by-case solutions for that. Maybe blobs could indeed be shared, or maybe they could be pruned/deleted over time, or maybe distinct apps would fetch distinct blobs. E.g. a PDF-sharing social network would store different kinds of blobs than "Ferment 2.0" would store.

Conclusion

Overall I'm happy to find a promising and simple solution for this problem which has given us lots of worries in the past. I hope you're also happy about it! Keen to discuss this further :)

User has chosen not to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
User has not chosen to be hosted publicly
@Rabble %nIdMXv/gji5clgU5J1u4Cs1pARbiuYAzHNCrcJDBouk=.sha256

Planetary iOS & desktop doesn’t use the ssb directory. nichoth is working on a web viewer based on ssb-db2 (open sourcing when it’s ready) and then following up with a web app that mixes cloud and local in browser db’s.

Part of me loves the multiple apps sharing the same db but it needs to be done with better structure around the indexes.

In using ssb-db2 we’ve discovered some non-Manyverse features needed to be added, which is great, just part of the growing process.

We’ve got a prototype of Planetary using @moderator but we’re not quite sure why to do with it or how it relates to ssb.

@mikey %cNVZeTZMj7xE0J8fQEY/BJIhj0yKX8CwAqi/DA1oFVw=.sha256

@andrestaltz yay, no more ~/.ssb! 🙆‍♀️

standards exist, let's please use them. (on linux: XDG for apps, FHS for systems)

Join Scuttlebutt now