%YENqKJvNLjJ3c1Iej7m0vGaf+x0TFhjnK1QYreg13Ck=.sha256ssb-npm updates and plans
I think my original assumption that a package name + version (+ dist-tag) + tarball blob id is enough for a registry server to serve to npm clients for them to resolve packages was not entirely correct. I have seen that npm sometimes does not install the dependencies, as in %OPhbNTJ..., which I think is because the registry server was not including the dependency information in the request to /<package>. It seemed to be working at first, and I thought that was because npm discovered the dependencies listed in package.json after fetching the tarball, and I still think it might be doing that sometimes, but it also could maybe have just been following package-lock.json to get the dependencies.
So, I changed ssb-npm-registry to fetch tarballs and extract the package.json data and serve it in the responses to request for /<package>, and allow enabling or disabling this behavior with a config option: %pzbKy/6.... At first I enabled it by default for npm v3 and yarn, but I think it might also be needed for npm > 3, so it would make sense to always enable it. npm clients ied and dep are not as affected by this because they make requests to /<package>/<range> instead of /<package> - the former returns only one package version as a response, so only one tarball has to be fetched, which will be the one that the client will fetch anyway. Therefore I enabled the fetchAll config option by default for all clients in %j9l4L2u....
I think it is not great to have to fetch all the tarballs for every version of a package in order to install any version of the package when using the most popular npm clients. For example. minbase is 300-500 KB each and currently at 4 releases so the total is about 1.5M. If more sodium-native modules are imported, this would also become more data than I would want to be fetched.
So it looks like it would be a good idea to denormalize ssb-npm's schema by putting the package.json data into messages, as npm-ssb did, so that it would not require fetching so many blobs to serve a package. I'm not sure if all package.json data is needed or if just the *[Dd]ependencies properties would be enough. Lots of packages are already published without this info, but these can be republished (or just let ssb-npm-registry fetch them). Also, migrating packages will result in publishing more messages since fewer packages' info will fit into each message. But perhaps migrating package trees will become less common/large as more packages are already on ssb-npm.
%Uo0uIO8...
semver, as far as I can tell, is a total order. It doesn't support concurrency. For example. If I publish foo v1.1.0, and someone else published foo v1.1.0 concurrently, anyone depending on the feature added by either of us could get breakage - unless they pin the specific version blob in package-lock.json - or there is extra information to disambiguate. Alternatively, version every public entity, or use accumulate-only versioning (actually a stronger variant: don't publish any new features (semver) to an existing package name).
To address versioning, I would like to add the needed extra disambiguating information by adding a property dependencyBranch to npm-packages mentions, and also a versionBranch property. A package name + version + a message id should then be sufficient to resolve packages - for future ssb-aware npm clients (and maybe hackable into non-ssb-aware clients with e.g. npm's session header).