p2p scuttlebot bootstrap installation

In %rmWxJuK... I added support to #ssb-npm-registry for bootstrapping installations. In %HvMKdic... I integrated this into #scuttlebot for #p2p #bootstrapping .

How it works:

ssb-npm-registry, a scuttlebot plugin, serves an npm registry server on localhost:8043. It also now serves a bootstrap snippet at http://localhost:8043/bootstrap, and listens by default on all addresses (but will only allow npm publish from localhost, of course). The bootstrap snippet is intended to be as small as possible, so that you can verify it by looking at it (e.g. on your friend's laptop screen). The bootstrap snippet fetches and verifies a bootstrap script via a blob, and executes it, passing it the address of a ssb-ws server (so that it can fetch blobs), and an npm command to install scuttlebot. The bootstrap script runs a local stub npm registry server and ssb-ws blobs server, and executes the given npm command, pointing npm to the local registry server.

The bootstrap script consists of ssb-npm-registry/bootstrap.js and package metadata for scuttlebot and its dependencies. The package metadata is generated by ssb-npm-registry/index.js from scuttlebot's package-lock.json file and a tarball of the scuttlebot package, which are retrieved from another plugin, scuttlebot/plugins/bootstrap.js. That plugin generates a deterministic tarball of the scuttlebot package as it exists on the filesystem. If the installation succeeds, the newly installed ssb-npm-registry server should then independently generate the same bootstrap script hash as was used to install it.

This bootstrapping process works using ssb blobs and the filesystem. It does not depend on ssb messages as read by #ssb-npm . Therefore, once you install scuttlebot with this method, the new installation should immediately be able to bootstrap more installations, even without having synced any messages.

How to use:

Install on a new machine

• Find a peer who is serving a http://[…]:8043/bootstrap snippet.
• Get the snippet and verify its contents with your peer.
• Execute the snippet.
• Visit http://localhost:8043/bootstrap and verify that it has the same hash as the original snippet you used. (If it doesn't, that is not necessarily bad, but it would be good for someone to investigate to see what changed).

Migrate an existing sbot git installation

• Fetch and check out the ssb-npm branch of %scuttlebot (currently 1d1d4415d5bd917d7d5c9ebf6b35a17f221c6378)
• Run npm install
• Run git diff to check if npm changed package-lock.json. npm might add back registry.npmjs.com URLs if it cached some package metadata. If that is the case, run rm -rf node_modules; npm install
• Restart sbot

Migrate an existing sbot npm installation (untested)

• Stop sbot
• Run sbot on alternate ssb-ws and npm-registry ports:
  sbot server --ws.port 8988 --npm.port 8044 &
• Find and download a bootstrap script. e.g.:
  sbot blobs.get '&he5FGzwlA+BQ4Hx861I4FJ/pgPSRYJtsxad+dVKr69A=.sha256' > /tmp/ssb-npm-bootstrap.js
• Run the bootstrap script to install sbot:
  node /tmp/ssb-npm-bootstrap.js --blobs-remote localhost:8988 -- npm install -g scuttlebot
• Restart sbot (on its regular ports)

Deployed now on my pub:

• http://104.236.22.57:8043/bootstrap
• http://[fc56:8313:1e14:1a50:0c01:0850:a53e:7127]:8043/bootstrap

and on my laptop (reachable with #cjdns when i am online):

• http://[fc8a:0c09:2d7b:248e:2a3a:5dfb:9e41:12bd]:8043/bootstrap

The top-level hash is currently &he5FGzw... (85ee451b3c2503e050e07c7ceb5238149fe980f491609b6cc5a77e7552abebd0)

More info about ssb-npm: %IdZKhhI...

@alanz that should be fine. but sbot will populate gossip.json again with data from pub messages. this is done in scuttlebot/plugins/gossip/init.js.

so if you want to gossip only with cjdns nodes, currently you have to put them manually in an array "seeds" in your sbot config, since we don't yet have a cjdns-specific gossip mode