Yeah a lof of these CI solutions are very much focused on cloud first. I did find this that looks promising. Very few dependencies.

And thanks for the benchmark.js link, we need something like that for the statistical aspects. I have added it to the README.md I did yesterday in the benchmarking-ci repo, where I have tried to lay out an implementation plan.

Its interesting that we arrived at more or less the same conclusion about how we decentralize the running part. I don't think the solution is complex. You just share the results in a git-ssb repo.