You are reading content from Scuttlebutt
@andrestaltz %3MgHK4Jj4Nvo6XIAsRjZnZ0udP5EVLZVeZU5t9LZZyg=.sha256
Re: %HJTzjmTcr

Yes it's good to review what should be done because some time has passed since the quest was specified. Like some of those are done already, through different means:

Should: implement some indicator in the app showing that replication is currently happening
... interaction responsiveness ... got better when the app is connected to pubs and has just been opened

I also think we should relax the requirement for throttling, but it seems we really need on/off for each connectivity mode. I don't agree with Dominic on this:

after the initial sync, most syncs arn't really that big

Because it really depends how often you open the app, and how good your internet connection is. If you open it every day with fast internet, it's okay. If you open it every month with slow internet, it's as bad as initial syncs are. In places without internet, the use case for the app will be syncing a lot of content every once in a while (every week?) through a slow internet connection or through Bluetooth peer-to-peer.

We also need to make the experience of initial sync better, because currently it's "wait for 8 hours for a lot of content to come in", and people may not even be that sure whether they want to commit time and bandwidth to download all that content and only after that start to explore the Scuttleverse. There must be a way of just sipping some content to begin with.

That's why this quest is about prioritising as well: can we just fetch hops=1 data to begin with, and if the user is interacting with the app, stop the replication and let them interact with it. When the app goes to the background, resume replication, etc. Once all the hops=1 feeds are replicated, proceed to replicate hops=2 feeds, still obeying the same pausing behavior if the user is actively interacting with the app. Etc until we replicate hops=N where N is in the config. Also, the user should be able to manually pause/resume the replication (it currently appears as a notification on Android, it's easy to add some buttons on that).

Some of these features are related to ssb-friends, some related to the replicate plugin, some related to EBT, etc. Makes it hard to separate the work.

But most of these concerns are covered by having a sane re-implementation of the gossip plugin, which I think is the bulk of this quest. A well built connection manager that is agnostic to connectivity mode, and makes it easy to pause/resume any connectivity mode, etc, will help us get to the goals we need. One example currently the gossip plugin parses addresses with its own functions or with ssb-ref, and often doesn't support anything other than ipv4 or ipv6. We would need to delegate the parsing to multiserver, which automatically tries all the parsing functions available and returns the first one that worked.

I'm not emphasizing the particulars here, whether it's throttling or back-pressure and which module are we touching. To clarify what "prioritise" means, it's more important that we can solve:

  • App developers can easily tweak the syncing (gossip/ebt/friends/replicate) behavior
  • Time to interact with content in the Scuttleverse after installing the app is reduced to just a few seconds (max 20 seconds?)
  • If the user wants to interact with the Scuttleverse while syncing is happening, syncing stops and we dedicate CPU to support UI interactions
Join Scuttlebutt now