engineering the social dynamics for massively collaborative open source

Continuing from %1ifwAYi... (cc @ansuz @noffle @corlock and others), I want to give some food for thought or design inspiration to build a GitHub alternative. This topic has been in my mind since I wrote https://staltz.com/open-source-without-maintainers.html , that was before the M$ acquisition, I've been thinking more about it.

I still want to explain my thoughts more carefully in a blog post, but I'll layout some sketches here before. I'd love to discuss this further and incorporate your opinions into my opinion and then into the blog post as well.

One article I want to write is somewhat separate from this topic, but a necessary intro: The Free Cyberspace Manifesto. Here's a bullet list draft of it:

• Free Software Movement was important
• However, that was before the internet became huge
• FSM concerns itself with freedom over software as controllers of the software
• The internet was the first foundational cyberspace
• I believe global connectivity was a major paradigm shift, it marked the start of the information age
• Connectivity is somewhat more important than Computing
• I don't want to undermine Computing, because Computing is required for Connectivity
• But Computing grew out of Electric Engineering
• Still, Computing is somewhat more important than Electricity, even though these two are not competing
• Similarly, Connectivity is more important than Computing, even though these two are not competing
• FSM is about freedom in Computing
• We need freedom in Connectivity
• Examples of libre cyberspaces: the web, email, USENET, Torrent
• Examples of proprietary cyberspaces: Twitter, Facebook, Slack, GitHub
• Freedom in Connectivity is more important than Freedom in Computing, because Connectivity > Computing
• Example: I can have a fully proprietary email client, but email cyberspace is free
• Also: fully GPL software in a proprietary cyberspace is powerless
• Example: pick any GPL Facebook or Twitter client mobile app
• Cyberspaces are systems consisting of both computers and people
• Computers are information processors, but cyberspaces are knowledge processors
• A cyberspace is a planet-scale "software" (netware?) consisting of both computers and people
• Freedom in cyberspace is therefore about freedom within the netware (contrasted with freedom over software)
• Cyberspace is a place you live in, Software is thing you control or use
• Free Cyberspace was kickstarted by John Perry Barlow's declaration
• That declaration established the rights of cybernauts, but not the means of achieving those rights
• Connectivity is enabled by Computing, so freedom in cyberspace is enabled by free software
• The Free Software movement therefore directly enables the Free Cyberspace movement
• Example: git as GPL is one component that enables migration away from GitHub
• (It would be much harder if git were closed source)
• Freedom over (controlling) software allows people to have freedom within cyberspace
• Centralization always implies hierarchy (the thing in the center is actually at the top of the hierarchy tree)
• Centralization has escaped scrutiny by de-emphasizing the root of the hierarchy tree
• Closed source is to software what Hierarchy is to cyberspace
• EOF

The above is important to the GitHub alternative discussion because GH is a proprietary cyberspace.

The other article I want to write is all about massively collaborative open source. Here's the bullet list draft of it:

• The GitHub model of collaboration is now engraved in the minds of many programmers
• It's not the only model, though. The Linux kernel is still coordinated through emails
• Even pull requests, "patches", are sent via plain text emails
• Although email is arguably a less attractive technology, it is a free cyberspace
• As a conclusion, the Linux kernel project is entirely unaffected by the GitHub acquisition
• However, interestingly, it's hard to say whether the Linux kernel is a free cyberspace, because it has a military-style chain of command with one general: Linus Torvalds
• It's anyway an interesting and important model of collaboration. The oldest git repo, as well.
• GH model:
  • repo: contains git, issues/PRs stack, releases
  • maintainer(s) of the repo
  • requesters: create issues and PRs. I'll call these simply "requests"
• The maintainer has 3 roles:
  • author git commits on the git history
  • manage (as in workplace "Manager" type of manage) the backlog of requests
  • publish releases (which consist of choosing a name for the project, sending to npm/maven/etc, managing changelog and version names)
• Let's scrutinize these 3 different hats/roles: Authoring, Requests management, Publishing and versioning
• Authoring
  • This is basically when the maintainer is writing git commits
  • There are two types of commits: Convergence and Extension
  • Convergence is merge commits or anything that incorporates other branches/forks
  • Extension is every other commit. We're extending the history of the codebase
  • Extension is not necessarily a divergence, but can be
  • (Every divergence/disagreement began as a naive extension)
  • (Divergence/disagreement is only between people, not between commits)
  • In GH model, convergence is usually through pull requests
  • Convergence could be different though
  • One example: I published and versioned my own fork of react-native-workers, but never sent a PR. Later garrettm found the fork and merged it into his repo https://github.com/garrettm/react-native-workers/commit/97dbe7917c2b079ec2bdd08625067910a5c0115c
  • Hints towards a different social dynamics for convergence
  • Hints towards splitting the Author role into two roles: the Extender role, the Converger role
  • Of course it will often be the case that Author = Extender+Converger
  • But open source collaboration is always voluntary, never obligation
  • It's a service in the original sense of service: altruism/selfless
  • Some people may volunteer for extension, others may volunteer for convergence
• Request management:
  • Bug reports, issues, pull requests are sent to an inbox
  • The inbox is a hierarchical cyberspace
  • The inbox is a community
  • GitHub Inc. at the (invisible) top of the hierarchy, repo maintainer(s) as the second top
  • This hierarchical power can be used for unfair decisions in the community
  • Email (as in mailing lists in the Linux kernel) is one alternative for community communication
  • Other free cyberspaces could be explored for community discussions
  • Requests are only "requests" because of hierarchy
  • In a free cyberspace, issues, bug reports, PRs, would be just public reports
  • No one should be obliged to work on any of those
  • Extension and convergence work would be volunteered to address those public reports
• Publishing and versioning
  • In the GH model, the maintainer should also npm publish or something equivalent
  • This involves three responsibilities: distribution, naming, versioning
  • Distribution is about choosing a package repository: npm or ssb-npm-registry, etc
  • Distribution has typically been location-addressed
  • Location-addressing, e.g. https://registry.npmjs.org encodes hierarchy, implies a hierarchical cyberspace
  • Distribution could be content-addressed, e.g. Dat or IPFS, so the package is in a free cyberspace
  • "Distributors" are people taking the voluntary responsibility of seeding the package
  • Naming is about giving an alias for the package, usable by the community to discuss public reports (issues, bug reports, PRs)
  • Naming is often just a human-friendly alias for a vague idea
  • Naming is hard, often is a game of digging through the semantics of a word, and how it matches a concept
  • Naming is actually Onomatology and a bit of Ontology
  • Versioning is used to reduce vagueness
  • E.g. react is a vague idea, react@16.3.0 is a specific package
  • Versioning has been mostly monotonic x.y.z numbers
  • One problem with monotonic versioning in a free cyberspace is that it encodes only one linear history
  • Versioning in a free cyberspace cannot be a monotonic code
  • Another problem, unrelated to freedom in cyberspace, is using monotonic versioning for communication
  • SemVer: failed in its own objectives
  • Quote: "What you can do is let Semantic Versioning provide you with a sane way to release and upgrade packages without having to roll new versions of dependent packages". Is a lie.
  • ComVer: https://github.com/staltz/comver is about admitting that versioning is mostly about compatibility
  • Spec-ulation talk by Rich Hickey
  • In that talk, Hickey talks about the contract of a package: what it requires (preconditions) and what it provides (postconditions)
  • "Change" is therefore the evolution of that contract
  • Compatibility is maintained when the preconditions are made more general OR the postconditions are made more specific
  • A contract X is said "compatible" with contract Y if the above happens
  • Versioning in a free cyberspace should be a way of encoding a directed acyclic graph (DAG) of library contracts
  • One naive approach is to formally write the contract, create a cryptohash of it, and then forks (extensions) of a project can then refer to that hash
  • This approach is imperfect, though, because compatible contracts would have different hashes
  • Maybe there is a crypto approach here where compatible contracts would yield the same identifier
  • Maybe formally writing the contract of a library is hopeless, maybe contracts will always be informal
  • Example: some of the contract can be encoded as a specification, some of it can be encoded as unit tests, some of it with property-based testing (generative, Haskell quickcheck style), or formal verification, but this is tending towards mathematical proofs of algorithms, which is hard
  • Conclusion: contracts are hard to enforce
  • Versioning will always (?) be soft guarantees of compatibility
  • But at least it seems clear now that versioning in a free cyberspace would not be monotonic x.y.z versioning, but instead it would (somehow!) communicate compatible contracts between package X and package Y
• Conclusion: in a free cyberspace, the social dynamics of open source collaboration would be voluntary with ephemeral roles, not permanent titles
• Roles: git Extenders, git Convergers, Reporters, Community gardening, Distributors, Namers (Onomatologists), and Contract Compatibility Analysts.
• We need to figure out a way of doing all of that without fixed hierarchies

#storystub

@arcfide Thanks for raising the topic of Funding. I was going to include that but forgot somehow.

I share your sentiment when it comes to free-riding and open source consumerism. I've also felt this, from maintaining a couple repos with several thousands of GH stars. You mentioned "It amounts to the enforced or expected slave labor of a group of the most talented individuals in order to benefit the least talented or motivated individuals." I fully agree this happens, because I have personally experienced that. That said, I clearly disagree with the proposed solutions you mentioned, but I hope that this disagreement is just "different thinking", not conflict. :)

I was supposed to mention the topic of Funding, and I view the overall open source movement (which includes Free Software movement) as having a culture of gifting and sharing. In the end, it creates an economic model based on voluntary donations, where donations are not just money, but also artifacts, code, community gardening, etc. Another economic model, the one we find in basic capitalism, is based on exchanges. What you described is basically exchange. In your words, "When someone delivers value into the collaborative ecosystem, they ought to receive meaningful compensation in return". The exchange economic model makes the world keep on going. It works. But so does the gifting economic model. Both are successful in their own qualities, to make the world spin.

I contend that the exchange model doesn't make a lot of sense for open source. Or for knowledge sharing, for that matter. E.g. Wikipedia as a knowledge sharing and collaboration model. Let's consider, e.g., active income and passive income, for bug fixes, new features, and new projects. Two dimensions: type of income versus type of work.

If you apply active income (fix this issue and earn this much money) for bug fixing or new features at scale, we begin optimizing for the wrong things, that is monolith projects, because bug fixing and new features at some point should stop if you have a small enough library, but workers have incentive to keep the project monolithic so that they have enough work in the long run. It basically removes incentive for small and modular packages. The other question is who should pay for that active income if many will share the benefits? Let's say Alice and Bob want bug 1850 fixed. Bob stays silent about it as long as possible. Alice loses her patience and ends up paying for it, then Bob reaps the benefits without paying anything.

Then, for other type of work such as new projects, who should pay for those crazy experimentations that end up becoming actually useful projects after 4 years? There are thousands of weird and unknown open source projects that end up nowhere, but some of them survive as really good and useful. No one can predict which one will turn out to be great. It's kind of like startup investment, except in that case there's equity involved, so it becomes a gambling game with expectations of high wins. But the winning dynamics don't quite apply to open source projects.

In the other dimension of passive income, it tends to become like royalties, and mostly applies post-success. A developer works for free many months/years on a particular project. Then stops, but their project is successful and keeps on bringing money through various sorts of fame-driven streams of income.

Apart from a few isolated use cases where trust is high enough between funder and worker and active income is applicable, I think at large scale the exchange-driven economy doesn't fit well with open source. I'm curious about what kind of models people can find for actually-fair exchange-driven open source income, I'm all ears. I'm also curious if https://licensezero.com/ will work out for anyone. But overall I believe it's like trying to screw with a hammer. Exchange-driven economy is not a good fit for open source.

What I've witnessed that works well, fits like a glove, for open source is the donation-driven economic model. I've been receiving donations from http://opencollective.com/cyclejs and it just works. Backers volunteer to give money. Contributors donate their time to fix issues. Others donate their time to write docs. Other donate their time to share knowledge at conferences and meetups. All these things add up, all these things are "work". Motivational psychology also tells that volunteering is a very productive and engaging activity, because it emphasizes autonomy (of task and method), has purpose, and does not reduce human effort into a do-this-earn-that activity (a reminiscent of the Industrial Revolution). This is why there are people who genuinely will work for free doing open source, even if you tell them they are being slaves. Because they feel they have purpose and they have autonomy to pursue it however they wish. There's nothing you can do to prevent people from making themselves "slaves" (unpejoratively). And we tend to apply this to the monolithic "maintainer" role, which I was proposing we split into multiple modular roles. The Converger, the Extender, the Community gardener, the Reported, the Seeder. All these as voluntary "slaves". Seeding torrents (or seeding Dat files) are also donations of bandwidth. Even bug reports are donation of information, when they are properly reported.

I've witnessed a lot of open source consumerism, but I believe its root come from the surrounding capitalistic urban culture (largely in America) that a lot of those entitled people live in, and not a product of the gifting culture in open source communities online.

@kik Thanks for sharing some history. I'm eager to read more history to understand what was going on. For instance, I'm curious if distro maintainers and the role split correlates to small modules or monolithic projects. I'm in general reading up on other material before understand what others have done in the past. For instance, before talking about free cyberspaces, I'm checking what the FSF, Stallman, and Eben Mogden have said about those topics. Etc.

I know it might be too large for blogposts, but I do blog posts, I don't care what I call them. They're documents distributed on the WWW. :) I don't want to make it look too formal that it would scare away some readers. That's why I'm also making some diagrams.

I agree we should experiment with new models and older models alike. Mix and match and see what happens. I believe that if something is easy to do, then it will statistically be the most common thing to do, and over time, it'll become the official thing, and people will start becoming blind to other ways of doing it. E.g. the GitHub model emphasizes the repo owner as the ultimate maintainer. It makes that easy. Which turned out to make the maintainer role official. If we manage to split those roles and make each one easy and seamless, then we will see the emergence of those roles. For instance, if p2p seeding is as easy as like buttons, then we will see the emergence of the Seeder role. On the other hand, it's substantially hard to seed npm packages independently from the official registry, and as a consequence we don't see a lot of that activity, which in turn means the role of independent package seeding does not exist in the npm community.

@alanz Also note that arcfide was talking about Free Software (I assume GPL) which would actually forbid companies to use it without making the company's code also libre (!). I think the free-riding of companies happens through non-free open source, such as MIT licensed (typically).

Archlinux and Gentoo forums are amazing : users know they are alone, and they always find a way. They'll discuss their problems, confirm them, find workaround, and sometime even share patches.

This is awesome! Thanks for telling @kik, I'm curious to go take a look at their community. It confirms somewhat that if you remove the expectation of "customer support", then people will volunteer to do the necessary work.

I'm curious to know what you all think of the Tidelift subscription model of funding open source.

It seems like they will be basically selling support/security contracts to big enterprises that the software maintainers will need to fulfill in exchange for a monthly payment, of which they will take a cut.

There are more details here: https://cdn2.hubspot.net/hubfs/4008838/Tidelift_LifterGuide_final.pdf

I'd prefer if everyone had their basic human needs met and the work and priorities could be guided by fun/community, but in absence of that, I hope this model can alleviate some of the free-riding and help ensure open infrastructure is funded on top of which we can build free cyberspaces.

@andrestaltz what do you think about the village issue tracker I've also had lots of thoughts about a "maintainer" role that is decoupled from authorship. Often someone writes a module that gets popular, but maintaining it is no longer fun, maybe the author doesn't even use it anymore (that's me with JSONStream). I don't care about "owning" this code, or the responsibility that entails. Could users just subjectively subscribe to who they want to maintain it? They could "elect" a forked maintainer - it wouldn't change "my" version - but it would end up being maintained by who ever cares to.

@dominic About teamware, sorry that I didn't comment earlier. I saw the link to the Harry Potter parody book, and that turned out to be an internet rabbit hole, I didn't read the book but ended up digging through Wikipedia a lot.

The ideas you put in that thread were a lot about affordance theory and UX design. A "join this org" button or "too hard for me" button is a message sent to the community. I think it would help. I think we need to experiment with different models and see what happens.

My bet is that if you create a public discussion forum, owned-by-none yet owned-by-all, people will figure out what needs to be done and by whom. I believe they'll self-organize into some structure that makes sense for them.

One thing I think would make a lot of difference is to separate discussion forum from forks graph. A GH repo's default view is files+readme pertaining to one fork of that project (consider that the origin repo is also a "fork"). What if the default view were forks-graph + readme? I'm thinking that emphasizing one fork alone is bad because it tends to centralize efforts on that fork. The fact that GH makes cross-fork collaboration hard (at least because there isn't cross-fork discussion forum) tends to centralize discussions in one fork. If all forks are displayed equally, and if it's easy to navigate and utilize/install other forks, I think we would gain a lot more flexibility.

A picture is so much better to explain these stuff. Here's what I mean, with screenshots I forged by modifying the DOM:

Cross-forks project view

• Owned by no one (it's noderify not dominictarr / noderify)
• Owner-less place contains the issues and PRs
• Shows you currently know graph of forks

Fork view

• Owned by dominictarr
• Has no issues / PRs
• Shows the files, commits, readme

Oh, and it seems like the fork with most "value" would naturally be the most common, and this has some nice similarities with blockchain: the fork with highest "proof of work" imbued into it turns out to be the community consensus. But for Git projects, this means it should be easy and swift to shift from one fork to another as soon as the other fork has more "proof of utility". Currently on GitHub it's not that common to use alternative forks that have higher proof of "proof of utility", simply because of naming and fame (and github stars).

Addendum # 3: I just realized that the "fork view" requires nothing else than rendering data from a git repo. It doesn't have any added metadata whatsoever. It's literally a read-only rendering of the git repo. That's actually quite useful! It means the cross-fork project view is the only git-less thing to build.

hey, don't have capacity to catch up on this mega thread right now.

My first question is : who has capacity and drive to build something in this neighbourhood?
(subtext - I am most interested in having this conversation with people who are intending to take steps, not so much in the theory conversation. While theory is useful, it's not that useful if there's only 2 people who can spend a half a day a week on a thing...)

@mix as far as I know, noffle is doing lots of stuff. see e.g. https://github.com/noffle/gitverse and it's hyperdb-based. I think this theoretical discussion is actually also practical. It's just a different type of practical: on the philosophical level. Which is also important because we shouldn't waste time building the wrong things in the right way in the illusion that building software is worth more than theoretical explorations. Another way to view this is that we're doing UX design here, which is important to happen before development starts.

@mix

@hsribei Thanks, I took a look at Tidelift. They had a couple good remarks. The core idea, though, is a middleman business model requiring Tidelift company, also it wouldn't apply well for small modular packages, mostly works for monolithic libraries.

Look forward to watching that talk @kik , excited to hear you're about building an issue tracker. I'm currently interested in the incarnation Dominic has been discussing - conceiving of it as a community space which happens to manage things which need doing (not a straight clone of gh issues, which are pretty 1D).

I'm totally into planning, don't get me wrong. I'm just strapped for time and I want to be part of this conversation but I'm a slow reader. Can't comit to reading this mega-thread, so was wanting to check if anyone was planning on moving on anything, because that's where I want to put my energy <3

cc @andrestaltz

@kik yeah this is something I really want to build, and think it is important enough to prioritize, but also have a bunch of other stuff. @dinosaur and I have done some pretty serious whiteboarding about it at least. I would be quite thrilled if someone else took the idea and ran with out though!

@arcfide I've also spent some time imagining a system where you declared the "credits owed" of the other work you built on, then if someone credits your work, some of that credit goes to the people you credited etc, etc. I'm not sure if there is any suitable payment technology available for this. Doing something like this with ordinary banking would be pretty hard to get legally compliant, and cryptocurrency doesn't solve the problem because it can't do micropayments.

I'm not really even sure if that is the answer, because the value you can create with open source isn't scarce. money is designed for scarse things... what ever these "open source points" might be should they really look like money?

Let's get on a call?

It looks like there's a noncompetitive arms race of ideas related to "p2p replacement for Github issues" and there are lots of overlapping ideas:

• Dominic's & Mikey's "Village" / teamware
• kik's drafts
• noffle's hypergit / gitverse repositories
• staltz's ideas

I think we would benefit from talking about this together in a mumble call, it's higher bandwidth of idea exchange than text.

It's unclear at this point how much of this will be on hyperdb, how much will be on SSB. Every time the point of Sybil attacks, community, following/blocking, and trust comes up, it seems pointless to try to re-solve this when SSB has already solved it. So there's potentially a lot that hyperdb can provide, as well as SSB can.

How about? @noffle @dinosaur @dominic @kik ?

I'm interested in joining a call. Can we get some timezone checks :

• dino / dominic / mix : new zealand
• noffle : pst ??
• andre : finland
• kik : ??

I vote for Tuesday morning in PST timezone (for @noffle ), which is Tuesday afternoon in Europe, but that's a bad time for NZ. I'll set up a doodle, but I need to sleep now

I'll be in hawaii next week, but keen for a call!

Back on (the original) topic, as I read through Git documentation, I see how it was built to work with email, and how from early on they thought about these different roles which now we just group into one thing, the "Maintainer".

What I said as "Converger" role was actually first named "Integrator": https://git-scm.com/docs/giteveryday#_integrator_a_id_integrator_a

Read this large design doc I made for Gitverse: https://gitlab.com/staltz/gitverse-ideas

Also available as

git clone hypergit://c8eef7ce2d668a2cd7c3aede6ac8f63b9512757eddd81bbb1a984d6538a72c40

I'd like to join in on a call, but I'm also located in CEST and any weekday will be difficult for me to attend.

I don't have any clear ideas to contribute, so it's probably ok if I just catch up after.

As a sidenote to the whole version control as a service, I think that people should look into Fossil because of the following reasons:

• It is the VCS created to maintain SQLite and has been in use by ages.
• The file format for the repository is SQLite
• License is BSD
• Built-in web interface
• Built-in Wiki
• Built-in Issue Tracker
• Built-in web pages serving
• One of my favorite features is that since the repo is a SQLite file, generating reports, findings things and cross-referencing commits is very easy and powerful.

When you clone a fossil repo, you clone not only the source but all the other assets as well. For example, when you clone the fossil own self-hosted repo, you end up with the source, the wiki, the homepage and the issues.

This can lead to a quite decentralized system if you want as there is no single server needed for those nice features. Also, since all that data is on your machine, it works offline. And one could build scripts to sync it over LAN or even over SSB. The sync protocol is quite simple and doesn't reinvent the wheel, it is just a text-based protocol over HTTP. Very easy to host and reimplement.

Why am I talking about this? Because I see our community doing the "Let's Reinvent the Wheel" a lot and I am quite fond of it. I think that we could use Fossil if we decide not to reinvent the wheel, but if we decide we must reinvent things, then Fossil serves as a good example of what can be done beyond the sphere of Git and GH clones. If this community ends up building some VCS as a Distributed Service, I'd like it to have Fossil qualities <3

@noffle I don't know, but I suspect it doesn't. The protocol is a line oriented one over HTTP and is actually only HTTP, the SSL part means have a proxy in between to handle crypto and let fossil receive plain old HTTP, which is kinda cheating but works very well. I can see it two paths for implementation of some interoperability with Fossil and SSB:

• Bridging HTTP to SSB, not unlike blob serving is done. In which an sbot plugin exchanges the needed HTTP requests with a fossil client but would probably require reimplementing large portions of fossil.

• By going from fossil to git and then to git-ssb as described in this guide for synchronizing fossil with git. It sounds quite hackish but it might actually work and require minimal plumbing. The advantage over git would be the included ticketing, wiki and site system.

Still, the main advantage for me is the benefit from storing things in SQLite. Repos are a single file, easy to transport and safe keep.

Still, all the above would be opaque in the feed, all the fossil stuff would be masked into git deltas that are not useful to the other SSB applications, I mean that even though fossil has wiki and tickets, that data would only be visible to the user after a repo is reassembled from its git deltas and checked out from fossil cli. I don't know if that is what we're aiming for.

We might want ticketing, wiki and web, to be visible in the feed in a usable way so that apps such as patchwork, patchbay and future hypothetical patchhub can see them. We might be better served by studying fossil and then replicating part of it in our ecosystem while keeping git-ssb as the file repository.

As some additional info about some fossil qualities I like, check out how easy it is to do complex queries (which we don't do by hand but fossil does on its own, there is more info about Fossil Vs Git from this page, just be aware that some points there are controversial and the author is biased, and don't forget to check this really nice page with reprt examples which provides quite nice project awareness for contributors.

@noffle I did a large reply before reading this message, I think that using a p2p proxy as a discovery tool and helping introducing fossil server to other peers might actually work... I haven't though about that at first, I was thinking more fossil to ssb, and not fossil to fossil with help from ssb. I like this.