ALL YOUR BUTTS

The gossip is too gossipy!

my understanding is that anyone can connect to any other pub or user, and as long as they can name a user, they can say "Hey, ahh, have you heard from @ye+QM09... recently?" And that computer will be like, "Oh you know @ye+QM09... ! Yeah dawg I got you, here's all their messages" ... which happen to include messages about other people.
Rinse / lather / repeat ==> you have all the humyn data connected from anyone connected to one another in any way

My explicit fear is the emergence of a Google-butt.

It starts as a really excellent service - Hey I notice you have a hard to navigate p2p web there... can I interest you in a map or guidebook to help you on your way? YES PLEASE.

Once the Google-butt is insinuated it's hard to get rid off because it's so gosh darn useful. Meanwhile the Google-butt has some overheads, but you know what, it's been spidering its way across the scuttle-pelago and it sure does have a lot of user data. It's got social graphs out the wazoo. Only a matter of time till your data is the product and things you've said are being batch processed by large institutions.

Gossip among friends

I'm at a party and some stranger walks up to me and we get talking. Turns out they're a person one of my friends has mentioned a bunch. We have a laugh at the serendipity. They ask how our mutual friend is and I give them the goss

Perhaps our scuttlebots should be more discerning. I want to see evidence that a person knows one of my friends before I starting spewing their infomation.

Possible ways to do this:

• only talk to strangers who have been followed by someone you know
• only talk to strangers who have been mentioned in your social circle
• expect total strangers to bring proof of friendship before you invite them them in to your party
  • a note with their signature on it!

Wait this makes the network even harder to navigate

Maybe not. I mean this is how my in person networks work and they work fine. There's no public index, it just works.

Could be that clear patterns for introducing friends makes it more personal, and build more of the connection that we want to see.

I personally still want to talk with strangers at open events. I'm not quite sure how that works. I think there are different sorts of follows. Like I follow you enough at an event to want to hear a bit about you, but that's not the same as recommending you to my friends dinner party. This might be another problem for ssb-actual-friends - it's easy to see if I only spoke to you for 5 minutes a year ago.... you're probably not a close friend. Meanwhile you can see I go to Art~Hack with @mikey every week

See also recently:

• The wisdom & madness of crowds
• Are groups the future of communities on ssb

Yes, good post @mix! I've been thinking about this a bit lately and posted an issue on the scuttlebot repo last week:

Pubs should have option to enforce gossiping only with peers they know

@mix this would be very simple to implement. there would be two parts: you'd need to have an sbot plugin that did something like:

sbot.auth.hook(function (fn, args) {
  var id = args[0] //the id which is trying to auth.
  var cb = args[1]
  checkIsWithInNHops(sbot.id, id, 2, function (err, within) {
     if(err || !isWithinHops) cb(new Error('stranger'))
     else {
       //fall through to other auth methods
       fn.apply(this, args)
     } 
  })
})

see also the same thing in the block plugin: https://github.com/ssbc/scuttlebot/blob/master/plugins/block.js#L57-L60

And then you'd need gossip to just not connect to pubs outside some hop radius.. We don't have a similarily simple way to change who you connect to there... but @piet is working on that and we mentioned this case.

oh maybe this is better example code, ssb-ws allows enough methods to run the liteclient if you are followed by the instance https://github.com/ssbc/ssb-ws/blob/transport/index.js#L54-L67

@Cole I'm interested in superceding follows with the actual lived situation. e.g. a person doesn't need to check if I've said I follow Cole to know if we're intimate. We're not yet because we've not been in that many conversations. This conversation and the fact that I talked back to you show that I'm incrementally closer now than I was before though!

for making pubs check if the inbound connection is within their friend hops before replicating data.

However, I don't think this will quite solve the issue. It seems to me that if you put data in public, someone will find a way to discover it.

I do think that @dominic's next step towards private groups is closer to a solution. It'd make random crawling of a message difficult for people who do not have the decryption key. However, you still have to make sure no one shares the decryption key.

This is a difficult issue to solve when you're publishing to an append-only log.

@masukomi - I value being able to find random people, communities and stuff too (on twitter, reddit, etc.) What if you could still fetch individual messages (via their ID), but just not request the full feed of someone you're not sufficiently close to on the follow graph?

That way, you could still discover new content by following links (either through manually following linked messages and finding new people and groups along the way) and fetching any individual messages you encounter outside your follow graph using ssb-ooo, but someone couldn't just go out and harvest the entire scuttleverse.

That way, a Google-butt could still emerge by crawling messages that are heavily linked to, but preserving the privacy of some 'less heavily connected / public' parts of the social graph.

Or perhaps they could, by finding a way to form friendships with enough random people in the scuttleverse through some incentive 'hey, befriend me for some free vouchers / discounts' - I'm sure there's some tricky scenarios to consider...

@masukomi

I guess I'm envisioning finding new content / islands through people linking to interesting posts or some functionality equivelent to retweeting. When you view these posts your client can then say 'give me that individual post that was linked / liked / retweeted by my friend please', but if you're not sufficiently networked with the person your client can't just say 'give me everything that randomer has ever said please.'

The mechanism of links / 'retweets' / likes could allow certain posts to 'go viral' and bring islands closer together, maybe.

Like you, I also like the ability to 'spider' my way across twitter myself (as though I was a web crawler), and find interesting things.. Like, look at a thread, look at the replies, look at the profiles of the repliers, see what they've been replying to, etc. However, I think I'm willing to sacrifice some of the ease of that experience to reduce the likelyhood of a google-butt emerging and embracing scuttlebutt, extending its functionality (by making it easier to find random content), then changing the protocol, then making the user the product again.

I'm not so worried about a google-butt that is just a proprietary and separate search engine application that makes content and people easier to find. I'm more concerned about a large-scale google-butt that is a proprietary patchwork-like client, makes the user the product (filtering content, promoting advertisements / more commercially valuable content, etc), or actively intervenes in the network with propaganda in a cambridge analytica style manner (leveraging its view of the entire network to see how successful its interventions have been.)

Leaning on ssb-ooo for spontaneous discovery is the best compromise between these two values I can think of... Perhaps we could also make some more voluntary mechanisms to allow randomers to discover each other in a controlled / more limited way. Or allowing clients to opt in to sharing their entire post history with spiders.

Or perhaps putting this limitation in just makes it a little bit harder to become a google-butt. Ugh, I don't know... It's a tough problem.

@masukomi isn't the whole idea of ssb-ooo that the peers you're requesting the post from will 'fan out' the request for the post (by post ID) until a peer is found that has it, then give it to you?

I've just realised that this idea probably wouldn't work anyway, because looking at the 'raw' version of your post, there's a 'previous' field (with the user's previous post on their feed), so a google-butt would just have to continually request the previous to get the majority of someone's feed and then have a bigger crawl space ( CC @Matt McKegg who I think also had the ssb-ooo idea.)

I don't think the 'previous' can just be omitted, because then the client can't verify the post wasn't forged, if my understanding is right.

Unpopular, unsolicited, and polemically one-sided opinion incoming: I think restrictive gossiping is a bad idea.

Why? It can not be enforced. Restrictive gossip is intended as a mechanism to keep data from being truly public, instead making it only visible in some sort of social sphere. But this a distributed system, once you send data to someone, you can't do more than nicely ask to not leak it. But at sufficient scale, these data leaks will be constantly happening anyways, whether it is through malicious behavior, well-intended efforts such as the public viewer, carelessness, laziness or plain bugs.

"But we have blocking, we will block these users." Sure, but that won't undo the data leak. And first you need to realize that data is being leaked. Somewhere among the thousand people of your foaf radius (which may or may not include popular pubs). And also, this directly conflicts with the premises behind out-of-order messages.

No matter what kind of promises any number of mainstream implementations make about publicity and crawlability of data, it does not change that fundamentally:

• every unencrypted message on ssb is public forever and to everyone
• every encrypted message on ssb will be public forever and to everyone, not today, but tomorrow someone may break the crypto

Talking about restrictive gossip does nothing more than closing the eyes and hoping the problem goes away. It won't. All this does is to give a misleading sense of security/privacy to nontechnical users, who will then suffer the hardest when data gets inevitably leaked.

A strong protocol should embrace [0] the realities of distributed settings, not willingly ignore them.

CC @mix @Piet @andrestaltz @cryptix

[0] In this case, this could for example mean using ephemeral public keys for replication to increase user privacy, or gossiping over a resilient random overlay network. But let's not get into that right now, I just want to bring up that there are indeed gains to be had from accepting the loss of control over data access that ssb inherently brings with it.

The way I see it, is there are two approaches to group messaging that are being discussed:

The first is cryptographically secured group messaging, using decryption links.

This is not implemented, but would be cool if it was. Does anyone know how to do this? Dominic said 'coming soon', so I assumed he had done some work on it.

The second is protocol-enforced group messaging, where you expect scuttlebot to only gossip with your friends.

I think what @aljoscha is saying is that trying to get protocol-enforced group messaging to work -- without encrypting the messages -- is impossible. The messages will just leak, because people will modify their scuttlebots to get messages from beyond their foaf range.

You gotta either encrypt the messages for specific people, or share a decryption key for a specific thread of messages -- otherwise the messages will just leak to whoever wants to read and append to them.

You can enforce not replicating a feed, but you can't enforce not leaking an unencrypted feed.

The only way to create real secret group messaging is with cryptography.

And then you gotta remember that all crypto will eventually be broken, so don't send anything that you don't want to leak in the future.

@keksdroid If the attacker here is say a Google scraper, I think the only way to defend against it is to encrypt messages. If Google wants the data, and it's unencrypted, it will find a way to consume it. There's no way to fight that on a protocol level or a social level, without encrypting the messages and sharing a decryption key.

I'd use decryption keys to send messages that could be unencrypted by a close group of friends, but I wouldn't expect the protocol to enforce that my messages aren't shared with Google.

A strong protocol should embrace [0] the realities of distributed settings, not willingly ignore them.

@Aljoscha there's more than one way to look at the "realities of distributed settings". 2 years ago i gave a talk on "nature's best practices for distributed systems" (video). the main point i learned from my research was that nature's primary strategy for security is the network topology, not sophisticated algorithms.

@keksdroid

An then maybe did this maliciously. In that case we don't need to talk because this is not part of the attack model.

So the security story is "Your data is completely private as longs as nobody evil tries to access at it"?

My previous post here was extremely opiniated/exaggerated for rhetoric, it stuck me as odd that those points have not been raised in the preceding discussion. I'm actually fine with doing these things, they are sensible and they will improve privacy a lot of the time. But I do stand by the opinion that the risks have to be communicated clearly. The moment someone tries to foster network growth by advertising the privacy of data in the scuttleverse, I will happily link to that post again.

@dinosaur Ssb has not yet found its preferred network topology. Maybe it never will - that might be a feature not a bug. But those things mean that we can not rely on protection through topology.

It is possible to design protocols that fully commit to creating a trustworthy topology. But ssb is curretly not one of them. While there is some vague talk about the relation between friendships and network topology, I have yet to see any clear indication that ssb plans to seriously focus on this (which would include making some sacrifices and increasing complexity). My mental model of "what ssb is about", built by observing application and protocol design, leans more towards open-mindedness than (borderline paranoid) web-of-trust enforcement. This very thread shows that a lot of people value the open-mindedness. But that means that relying on the trustworthiness of the network topology is risky.

Nature's solutions often deemphasize the value of individual entities for the greater good. But once we are talking about settings where the entities are fellow humans, this becomes tricky. We can't just say "Oh yeah, a few people might <insert arbitrarily bad scenario here> because of our choices, but on average these choices make the system flourish.". There might be die-hard utilitarians who don't see ethical problems there, but I'm definitely not one of them.

@Aljoscha (this is starting to ramble so feel free to ignore)

to me, Scuttlebutt has always been intentionally focused on small-world network topologies. however, we haven't been able to yet reach the small-world networks ideal, i reckon mostly because pubs have dominated (and centralized) the social graph, hence "rethinking pubs as a second device", new user invites, why i'm against public pubs, and why i want to make it easy for anybody to host their own pub.

on another note, i agree it's silly to think people won't be selfish, however i'd say Scuttlebutt is about exploring the game theory of infinitely-repeated games with non-repudiable messages. if you defect, it should be widely known that you defected, so other agents can respond appropriately (by defecting against you). if we can't coordinate our social networks against bad actors then how do we expect to coordinate to solve societal problems?

@dinosaur (happy to ramble along)

to me, Scuttlebutt has always been intentionally focused on small-world network topologies.

Trust is highest in highly clustered areas (i.e. within a single social circle), yet efficient and resilient replication wants to bridge between clusters, that is it wants to follow the paths of least trust. There's an inherent tension there, and no optimal solutions. This is among the sacrifices I mentioned about focusing on maximizing trust during replication.

I lean towards prioritizing replication resilience and efficiency, and so does Dominic, I think. Which makes me so wary about relying on the trust guarantees of the network.

If I were to break compatibility and redesign ssb completely from scatch, I'd use a peer sampling service on a fully random overlay network, and then use a quality metric (primarily) based on feed subscription overlap to prioritize replication connections (and also do some clever scheme of gossiping network addresses based on the quality metric). That is pretty much throwing all the trust overboard, instead optimizing for replication quality. Actual ssb is much more moderate, and that's probably a good thing.

I love unpopular unsolicited polemic opinions, it forces us to think honestly and avoid self-reinforcing group thinking.

Talking about restrictive gossip does nothing more than closing the eyes and hoping the problem goes away. It won't. All this does is to give a misleading sense of security/privacy to nontechnical users, who will then suffer the hardest when data gets inevitably leaked.

What is "the problem"? Is it "fake privacy", vulnerability to leaks? Is it weak replication resilience between clusters (seems like the opposite problem of small-world privacy)? Some of these problems are specific to some cultures, and should have configurable solutions. E.g. I know that germans are very concerned about privacy, yet americans not that much. Americans are very concerned about censoring the bad people, other cultures (scandinavians?) aren't that much. We shouldn't promote a culturally specific problem to global scale. I'm not so concerned about culturally specific problems as long as they have configurable solutions (e.g. hops settings, restricting gossip, etc). I'm more concerned about the global problems in Scuttlebutt, and those surely exist, and I think they are related to the issues Aljoscha is raising.

I've been thinking a lot about forgetability in Scuttlebutt, and whether we actually need sigchained messages for unforgeability. If you read up the security properties of secure-scuttlebutt and the sigchains in specific, it says:

Security properties

Secure-scuttlebutt maintains useful security properties even when it is connected to a malicious secure-scuttlebutt database. This makes it ideal as a store for peer-to-peer applications.

Imagine that we want to read from a feed for which we know the identity, but we're connected to a malicious secure-scuttlebutt instance. As long as the malicious database does not have the private key:

• The malicious database cannot create a new feed with the same identifier
• The malicious database cannot write new fake messages to the feed
• The malicious database cannot reorder the messages in the feed
• The malicious database cannot send us a new copy of the feed that omits messages from the middle
• The malicious database can refuse to send us the feed, or only send us the first N messages in the feed
• Messages may optionally be encrypted. See test/end-to-end.js.

See the part in bold. It assumes the peer is untrusted (malicious)! This thread was started by Mix recommending to only gossip with trusted peers. So by definition, you would not connect to malicious peers attempting to write fake messages, or reorder, or omit. This to me is very interesting, because I think as we start to recognize Scuttebutt's unique strengths, we also begin to recognize those YAGNIs that we couldn't have foreseen since the beginning. For instance, I believe we should transition to a much less Pub-centric topology, we should explore Rooms ("dataless Pubs with ssb-tunnel installed") and DHTs for relay P2P. Connect only to actual friends. I also am started to suspect that we should get rid of unforgeability guarantees of the sigchain, because it would buy us forgetability, ephemerality, which are very natural for human interactions, and one of our goals and principles is to have the technical layer reflect the social layer in the most natural way possible.

... continues

So perhaps "the problem" that you highlighted could be summarized as "the unintended consequences of unforgetable feeds". I always try to look at these problems from the sociotechnical perspective: without technology, how would human communities function? Could we replicate that in technology as accurately as we can? I view computer technology as just another language. Protocols are languages that computers speak on behalf of humans, so the issue is about designing languages for conversations that create communities. Before I digress into philosophy, let's go back:

Is the following sentence true? "Every sentence that someone says in real life will eventually be recorded and shared to the whole world". I don't think so, so neither should its Scuttlebutt counterpart be true: "Every unencrypted message on ssb is public forever and to everyone". Most friends forget what other friends said, except some specific messages. We should support the same.

Let's move on the next quote, is the following sentence true? "We should optimize for humans to spread information between each other, on a stranger-to-stranger basis, as efficiently as possible" I don't think so, so neither should its Scuttlebutt counterpart be true:

If I were to break compatibility and redesign ssb completely from scatch, I'd use a peer sampling service on a fully random overlay network, and then use a quality metric (primarily) based on feed subscription overlap to prioritize replication connections (and also do some clever scheme of gossiping network addresses based on the quality metric). That is pretty much throwing all the trust overboard, instead optimizing for replication quality. Actual ssb is much more moderate, and that's probably a good thing.

Trust means that some kinds of messages are spread more quickly in clusters of high trust concentration, and leak outwards slowly or with less efficiently. This is a feature, not a bug. To attack a cluster of high trust, you need capabilities that can match it. By not being able to be part of the cluster, a stranger cannot "catch" up with their information flow that well. The trust bubble protects those inside it. This mirrors the real world case where an outsider can have a rough idea of what's going on in a group, but if they try to attack the group by making accusing claims, they'll just be guessing. The attacker doesn't have the whole context to understand the group.

Trust-driven connections optimize the information flows to have higher bandwidth in clusters, but to have slow but positive bandwidth for outsiders. I think this accurately reflects the real world too. Imagine an isolated village somewhere in India, there might be lots of interaction and stories going on there. Something eventful may happen, and that's when the outside world gets interested in the event, and comes to inquiry what happened. They get some information. They understand some of it. And they report what they understood, which may be wildly inaccurate, because it's not as bandwidth-privileged as an insider is. This is also why spies exist, to provide a high bandwidth outside channel into a cluster, and it's interesting that spies are playing a trust game first and foremost.

Some social issues will never disappear with technology, because technology and social are orthogonal, you shouldn't solve one with the other and vice-versa. We can only hope to build technology that is not trying to subvert or alter the social, because when it does, then technology plays an active role in shaping culture, and ends up creating new social problems (e.g. Twitter harassment) although it has solved some social problems (e.g. Twitter reach and ease of engaging with any community). I think the technology layer should aim to harmonize with the inner workings of social info bandwidth, so that our answer to social problems are social solutions.

(Note 1: Statements in bold were used just to make it easier to skim and speed-read, not for shouting. Note 2: for every declarative sentence I wrote you can implicitly add "in my humble opinion" in front. )

@andrestaltz Those are some interesting thoughts. I don't think I will be able to take the time today to do them justice, but here are my most immediate reactions:

Is the following sentence true? "Every sentence that someone says in real life will eventually be recorded and shared to the whole world".

No it isn't. But what about this one: "Every sentence that someone says in real life can be recorded and shared to the whole world"?
That is the situation you have whenever you write data over the network.

As a corollary:

So perhaps "the problem" that you highlighted could be summarized as "the unintended consequences of unforgetable feeds".

I'd rather summarize it as "the inevitable consequences of sharing data over a network without assuming 100% trustworthiness".

To attack a cluster of high trust, you need capabilities that can match it.

There's another kind of attack: isolation. If network connections mostly stay inside clusters, it becomes really easy to create network partitions, whether through attack or everyday network failures.

One lens I'm using when looking at a gossip protocol is how it can deal with eclipse attacks. Trustless solutions are both complicated and resource-intensive, trust can help a lot there.

@Aljoscha

"Every sentence that someone says in real life can be recorded and shared to the whole world"?

Good point, this sentence is roughly false in the social realm, but true in the technical realm. I think in that case we just attribute it to computers having perfect recall memory, whereas people have non-persistent memory. Two caveats though, one caveat that favors the equivalence social-technical, and another caveat that disfavors Scuttlebutt. (1) Every sentence someones says in real life could be recorded with hidden recorders (although these are also technology, strictly speaking). (2) sentences in Scuttlebutt are cryptographically signed, whereas real world conversations are not (or are they? Physically seeing a friend speak is a "signature" for their sentences). Anyway, lots of caveats of caveats to unfold here.

On isolation and network partitions, it's a problem or it's a feature, depending who you're talking to. So it's a social thing more than it is a technological thing.

E.g. I know that germans are very concerned about privacy, yet americans not that much. Americans are very concerned about censoring the bad people, other cultures (scandinavians?) aren't that much. - @andrestaltz

If you're going to bring nationalism into this, which I don't think is necessary, I wouldn't generalize so much. As an American I am very concerned about privacy and security, and that's why i'm arguing that you can't have private group messaging without cryptography.

While it's true that most Americans will take the "I have nothing to hide" argument when asked about whether they appreciate privacy, we also have to understand that many of the people living in my country and powerless to stop the terrible things that Silicon Valley and our government is doing.

This doesn't mean we, as a culture, do not value privacy and freedom of speech. In fact, both are written into our constitution.

Anyway, if we're going to generalize about nations, let's not generalize too much. There are many perspectives over here, not just the terrible perspective represented by Silicon Valley.

Continuing to ramble (cc @dinosaur, @andrestaltz), here's a small idea for handling trust in a gossip protocol:

Peer discovery happens via view shuffling as usual for gossip protocol, that is to say: A pair of peers with a direct connection established between them will occasionally exchange the ip addresses of other peers they are connected to. That way, both learn of the existence of potential gossiping partners, this process builds the overlay network (this is simplified, but provides the gist of it. In particular, ip addresses are just a stand-in for more realistic connection information). Now instead of solely exchanging ip addresses, you'd exchange triples of (public id, ip address, trust byte). The public id is something like the pubkey of a feed in ssb. The trust byte signifies how much trust you put into that public id, the lower the value, the more trusted it is. This trust byte gets modified as follows:

• when sending a well-known id, set the trust byte to whatever trust value you associate with it. Examples:
  • your own id is well-known to yourself and you fully trust it, thus send a trust byte of 0
  • if you happen to have built a follow-graph like most ssb servers do, and the id known to you because it appears in the follow-graph, set the trust byte to the distance from between yourself and that id in the graph
  • if the id is in the map of temporarily known ids (see below), retrieve the associated trust byte and increment it (saturating at 127, not wrapping around)

When receiving an id that is not well-known to you, add it to a map of temporarily known ids. As the value associated with that id, put in either the trust byte associated with the id when you received it, or the trust byte of the peer who sent you that id, whichever is higher.

This little scheme has very low overhead, but established a nice, distributed, and fully subjective notion of trust. In particular, any attempt to "cheat" to make something look more trustworthy only works on those who already trust you. But as an untrusted peer, you don't get any shortcuts to get others to trust yourself or some other id of your choice.

On top of this, you could do all sorts of useful stuff, e.g.

• only replicate with peers below a certain trust threshold t
• only pass on data to the application layer that comes from an id below a certain trust threshold t
• strive to always keep up at least n connections to peers below a certain trust threshold t, guarding against eclipse attacks, while at the same time using the other connection slots for less-trusted peers to reduce the likelihood of network partition

I've neen wrestling with this problem for a little while. My aim was that I wanted to make it harder for someone to map the whole network and read everything that people post publicly. The aim was not to provide any kind of false security.

The first problem is that anyone can connect to a pub and just start downloading feeds. For this I have created ssb-incoming-guard git-ssb, a plugin for scuttlebot, that you can install on a pub and then it only allows incoming connections within hops of the pub. It still allows anyone to use invites.

The second problem is that when doing outgoing connections, it should be possible to only connect to users and pubs within your hops. This is closely related to the work @piet is doing on replication, so I'm awaiting the outcome of that so see how an option can be added for this.