Who knows what would happen if all the timestamp-opponents (and I'm definitely not the only one) coordinated to simultaneously start randomizing [timestamps]. Would that divide scuttleville, or would the popular clients adapt? I will not try to start this experiment, but I would definitely be in the randomization camp.

Out of curiosity, could the current client devs estimate how user experience would be affected if some subset of users (say about five people you follow directly) started posting messages with random timestamps?

CC @mix @Matt McKegg @cel @SoapDog @Tim Schumacher @christianbundy @ev @arj @happy0

Patchwork uses both the sync time and the user timestamp to decide viewing order. The oldest stamp wins. So if people were to start posting things as far in the past, no one would see them. Maybe based on that selection pressure, they might stop doing that. There is no advantage to posting things in the future except that syncing someone new would put them at the top of your feed for that moment. At which point maybe you would block them?

Sorry for the cheeky quips in there, but hopefully that answers the question in regards to patchwork.

Are sequence numbers of messages taken into account at all? I.e. always order messages from the same feed in sequence number order, use timestamps to establish an ordering between two messages from different feeds?

The cheeky quips are totally fine, it's not like I haven't been handing out a lot of them myself over the past few days =P I feel like I should mention though that timestamp randomization can have benign motives, such as anonymization of timezones (and computer activity in general). You might trust your immediate peers not to keep track of when they received new messages from you, but you can't trust any future crawlers of your immutable log. At some point, there will be communities of people who want to keep that information private, and they would have to chose another client (and also they'd be isolated from more privileged communities).

depends on what you're viewing.
some views use received time, some views use ssb-sort to do a "causal sort" which takes into account timestamps (receved, asserted published) AND backlinks to other messages (which place a bound on when that message happened)

I don't use sequence numbers unless in ticktack and you're re-syncing your feed and the feeds of people you follow from a backup state

The sequence ordering is used on the profile page, but I think that is all. As @mix says, the branch value is used to determine thread order in patchcore.

@moid Is there anything about that use case that could not be done by adding timestamps to the message data rather than the meta data?

You might trust your immediate peers not to keep track of when they received new messages from you [...]

I just realized that sbot does track exactly that, as it provides methods to query based on receive order. Or does it only store the relative order instead of local timestamps (local timestamps are a completely fine thing to use btw, it's only global ones that are problematic)?

@moid unfortunately, we already made timestamps non-monotonic I was the only one arguing for them.

@aljoscha you can query by recieve time. You could also query by log offset (relative order). local receive time was the only api available before flume, and when I refactored flume in, I continued to support it so that clients wouldn't need rewriting.

log offset is used internally by views (because you can do O(1) reads of the log), and clients could use it I don't think any do currently.

I have at times waved my hands and talked about a "social proof" of timestamps. Basically, detect the causal order of all interactions between feeds. like the ssb-sort used by threads, but on every message. A large proportion of messages are replies or likes, so the partial order would be fairly tight. Also, since a hash link proves that the hashed message was created before the linking message, I think you could use this to estimate bounds for the timestamp. This would be useful because you'd get consistency (after a fuzzy period). I havn't actually sat down and figured out how such an algorithm would work though.

Here's such a thread: %hoAVki/…

I miss @Sybil =(

A quick summary of these things (mostly to sort this out for myself):

Mathematically speaking, sigchain backlinks define a total order on all messages of the same feed.

Subjective opinion: All chronological sorting of messages from the same feed should use this total order.

It is impossible to get a total order on messages from more than one feed.

Cypherlinks (assuming collisions are impossible ) define a strict partial order on the set of all messages. This is the largest happened-before relation for which ssb provides cryptographic guarantees.

Subjective opinion: All chronological sorting on messages from multiple feeds should be based on this relation. Receive-time or timestamps can be used as tie-breakers (which are needed since the order is only partial), but so could be alphanumeric comparison, or any other mechanism you can come up with. Since all tiebreakers are equal, none of them should be part of the core protocol.

Given a partial order (i.e. the happens-before relation defined by cypherlinks (which includes sigchain backlinks)), timestamps can be checked against it. Given two messages m1 and m2 with timestamps t1 and t2 respectively, if (m1, m2) is an element of the cryptographically verified partial order (i.e. m1 happened-before m2), but t1 > t2, then the timestamps are inconsistent. If t1 < t2 and (m1, m2) is in the partial order, then the timestamps may be made up, or they may not be. The very notion of a "correct" timestamp is useless in a distributed setting. The best you can get is "does not violate the happens-before relation", but that's all.

Even if a timestamp is from the future (from your perspective), this does not imply a malicious lie, but can simply be the result of clockdrift (either on your end, on the peer's end, or on both ends), or any other sort of issues (hardware failure, sandboxing, timezones, etc.).

This is not a hypothetical and theoretical issue, it happens all the time. I can remember multiple times where I've received messages "from the future" in patchwork.

All of these considerations are related to cryptographically verified relations. You can go on and use a web of trust to get probabilistic guarantees for the quality of timestamps, such as in the thread @kas linked, and there are surely some great "solutions" that involve proof of work. I'm personally not really interested in these approaches, at least not until we get the cryptographically secure parts done right.

@Aljoscha Right now mvd throws away messages from the future, but I plan on adopting the strategy @matt is using in Patchwork.

@Aljoscha I have no idea what happens to be honest.

Nice summary @Aljoscha

here's an idea : outsource TIME as a matter of trust to some peer(s)

e.g. an always online peer runs an atomic clock, and every 5 minutes it posts a tick message. Any peer who needs or wants time can backlink these tick messages in order to attach time to their thangs. This will stop things being cast into the future at least, and feed sequences and other users interacting will help locate a thing in time (if you want to check) making it less possible to cast something back into the past.

Anyway, you can trust the tick peer or not, or maybe you can subscribe to a range of them.
At least it gets the timestamp requirement out of the core protocol?

@masukomi I'm pretty sure a machine-learning person can write a fun little program that infers timezone, sleeping patterns, working times, travel, etc. with a frightening accuracy, given a feed with enough messages. Not everyone will want this to be possible. If clients encourage blocking in case of timestamp randomization, then these people are excluded from ssb, or at least all communities that use those clients. Which seems to currently be all of them, except patchfoo.

i'm not understanding what downsides I'm not seeing.

Timestamps don't work in a distributed system. Things break if you rely on them. Physically speaking, it is impossible for two entities at different locations to agree on the same notion of "now". More relevant for us, ssb needs to account for some byzantine faults, i.e. peers may lie about the timestamps, or they might perceive a different time than you, due to clock drift, virtualization, or frankly bugs. Every once in a while, a newcomer on ssb asks why some post is from the future, or not showing, or why posts got reordered, etc. These are not bugs that can be fixed, these are consequences of client devs relying on timestamps. It just does not work, yet the protocol encourages to use them.

I could not find a single good explanation that goes into detail of why the notion of distributed global time is impossible and undesirable, so here are a few different takes on it instead:

• tiny bit of physical background
• CS education intro
• paper by Lamport (who fortunately is a great writer)
• blog post about a real-world example

@mix Interesting idea. I want to stress that this does more than just "getting the timestamp requirement out of the core protocol": The core protocol can make no cryptographic guarantees about timestamps, but your suggestion does. And also, it is opt-in rather than mandatory.

Addendum to @mix: There are literally decades of academic research on this problem, as well as real-world engineering. If someone is serious about high-quality timestamps (with a specific model of error tolerance, be it in accuracy, trust, or something else), they can get them. They won't even have to come up with a solution themselves.

Good point @Aljoscha, it makes sense this is a problem that's had a lot of attention. This seems like an instance where we could be eating some nice store bought pate, rather than rolling and eating our own dogfood.

It would be cool to be able to fund a literature review of this topic

@masukomi

are there any real downsides to having them if you don't use them for ordering?

If there are timestamps, then people will use them for ordering. Look at the current client situation, Cel is the only one who does not rely on them (unless you specifically tell patchfoo to do it). Optional timestamps at least force them to consider it a partial ordering.

Then there's the privacy concerns I listed above. And just as a general approach to design, trying to answer "why" rather than "why not?". (I'd like to link a quote here, but I'm offline. The best source I can give is to search the "quotes" section of bret victors's website for "lua")

@cft

[...] which seems to have been overlooked.

Nah, just discussed elsewhere.

With a cypherlink-only approach, the query interface in createHistoryStream would have to change.

Can you elaborate on this? createHistoryStream only returns messages from a single feed, and on those backlinks give us an total order (irreflexive), with or without any sequence numbers. So I don't see how createHistoryStream would be affected at all by removing sequence numbers.
Note that createHistoryStream is used for (legacy, non-ebt) replication, not primarily as an API for clients.

get up to N causally-youngest log entries (the heads so to speak). Ideally only one entry exists, but better prepare for tangles where there can be multiple tips.

Ssb forbids those. A forked feed does not get replicated at all.

best documentation on EBT: WIP paper and discussion thread