OH JEEZ ... i followed someone and it put me in range of @sbot

their latest message : "sequence": 373837, (I'm one of the older and more active accounts here and I'm at 24739

loading them in just all but crashed my system, it was like another onboarding. lords, who does @vYqLJ+S8RSwrgU6Nxja0kM3d19oWqjv9Og2JCbDd8+U=.ed25519 belong to and how did I get in range of them I wonder.

I blocked them, but I fear they're gonna hit everyone else as well. Good luck !

umm @christianbundy how was that account scrubbing coming along?

on a sidenote, that @sbot seemed to be doing some really interesting stuff. I'll post the content of a couple of messages for those curious but not masochistic:

{
  "type": "user",
  "payload": {
    "msg_name": "game_join",
    "msg_data": {
      "roleId": 1199,
      "roomId": "00002"
    },
    "address": "TABwrK9wy3uFserWi55P9GucEkuHx4WH7P",
    "signature": "3752e19182367e08d37cf241645d27a40c181e1c2c6d3575fd92c8c1a6f80452",
    "transactions": [
      {
        "from": "TABwrK9wy3uFserWi55P9GucEkuHx4WH7P",
        "to": "OPERATOR",
        "value": 10,
        "token": 0
      }
    ]
  },
  "nonce": 316269,
  "app_id": 13
}

{
  "type": "operator",
  "payload": {
    "msg_name": "game_clear",
    "msg_data": {
      "roleId": 1199
    },
    "address": "TABwrK9wy3uFserWi55P9GucEkuHx4WH7P"
  },
  "nonce": 316262,
  "app_id": 13
}

{
  "type": "operator",
  "payload": {
    "msg_name": "game_finish",
    "msg_data": {
      "seed": 1552998979365,
      "fightId": 1,
      "rolesData": [
        {
          "roomId": "00002",
          "roleId": 1199,
          "token": "0.30000000000000004",
          "amount": "7.3000000000000025",
          "deadCount": 1,
          "reliveCount": 1,
          "timeGap": 156
        },
        {
          "roomId": "00002",
          "roleId": 1198,
          "token": "0.2",
          "amount": "-1.0",
          "deadCount": 1,
          "reliveCount": 0,
          "timeGap": 11
        },
        {
          "roomId": "00002",
          "roleId": 1189,
          "token": "0.4",
          "amount": "-2.0",
          "deadCount": 3,
          "reliveCount": 2,
          "timeGap": 147
        },
        {
          "roomId": "00002",
          "roleId": 1200,
          "token": "1.1",
          "amount": "-5.5",
          "deadCount": 10,
          "reliveCount": 9,
          "timeGap": 156
        }
      ]
    },
    "address": "OPERATOR",
    "transactions": [
      {
        "from": "OPERATOR",
        "to": "TABwrK9wy3uFserWi55P9GucEkuHx4WH7P",
        "value": 22.300000000000004,
        "token": 0
      },
      {
        "from": "OPERATOR",
        "to": "TABwrK9wy3uFserWi55P9GucEkuHx4WH7P",
        "value": 0,
        "token": 0.30000000000000004
      },
      {
        "from": "OPERATOR",
        "to": "TDVQCriSkfLJt4fY6vZKT3MePMhbL6uGhX",
        "value": 0.5,
        "token": 0
      },
      // .... more of this
        "from": "OPERATOR",
        "to": "TDVQCriSkfLJt4fY6vZKT3MePMhbL6uGhX",
        "value": 0.25,
        "token": 0
      }
    ]
  },
  "nonce": 316258,
  "app_id": 13
}

@mix,

how did I get in range of them I wonder.

You are friends with @TE5ys+3... who is following @sbot.

thanks @kas ! I couldn't open sbot's profile page reliably in patchbay to see their follows because of a post graph which explodes when the number of posts you've received from a person in the last 3 months is in the over 100,000 ...

Just made the same misstake, followed @Andrei Cociuba and my client blew up. I'm trying to block Sbot, but that message is on a queue far away.

Gonna be interesting to see what happens with my visual network graph after this.

ok, unfollowed @sbot.

can someone explain to me like i am a 5 year old adhd with tech knowledge why following that account is a bad thing?

@Andrei Cociuba by following an account you will propagate their feed to those who follow you, and not just to your direct followers, but also followers of your followers, etc (depending on the number of hops that each client is set to). If an account (like that sbot) published a large number of messages over time, when another client receives a propagation of that feed for the first time it will completely fill their view with those messages, basically acting like a denial of service. Some clients may handle this more gracefully than others in terms of processing, indexing, and UI, but invariably it will have an impact. This is why an often used "solution" is to block the account in question (i.e. sbot) or unfollow the account(s) that follow it.

This is not an issue specific to bots. I recently experienced a similar thing when a person who has been using ssb for a few months but isolated, i.e. in an "island" of their own away from the main scuttleverse "continent", peered with someone connected to the continent. This meant their whole feed was propagated at once to anyone following the peer in common, and resulted in my view being completely flooded with that feed. And this was a human generated feed. So you can imagine the impact of a bot feed.

I'm sure clients can and will do a better job of handling these major feed merges going forward.

hm.... prickly problem, indeed. sounds like something the technology could solve, if you dont really stop to think about it. any possible technical solution i can immagine will probably affect legit use way too much.

in principle i was aware of that effect, but i considered it more academical than anything. i didint immaigne i could crash clients and somesuch.

well, i blocked them now, so hopefully that will help.

hah it's no problem really, Sbot was on the edge of a community otherwise completely detached from me. so your follow actually folded 2 bubbles together ;)

Just ran a simulation in gephi on my existing local database.
Seems I mostly just got small loose unconnected bits of posts and a small batch of new users into my network just before the block started working ;) I picked up a total of 1200 new accounts in my database.

Pic of sbot (and you @Andrei Cociuba)

@mix, I was bitten by @sbot myself a couple of months ago, but managed to block it when it had ‘only’ reached sequence 77′404. It sucks when a client crashes on something as simple as this.

here's how to block from the command line (what I had to do)

$ ssb-server publish --type contact --contact "@vYqLJ+S8RSwrgU6Nxja0kM3d19oWqjv9Og2JCbDd8+U=.ed25519" --blocking true

This merging of bubble immediately added about ~250 MB of messages to my log... jumped from sequence ~700k to about 1000k. Kinda unreal, and a really awesome test case.

Nice rendering @Hendrik Peter. I think it's useful with Gephi to only show accounts which have more than say 50-100 messages to their name. It makes the graph more "realistic" to what's active. I also think adding data about how many messages a peer has posted as the size of a node helps grok the impact. I wonder who the accounts around @sbot are.... and I'm really curious what that project is

p.s. @Andrei Cociuba this isn't your fault at all, if anything it's really good to have these unexpected events occur to help us better understand the nature of this space!

Didn't @Christian Bundy had a utility to prune a log to remove a given profile? That would be handy right now...

so (given there's not documentation for this), can you confirm:

$ feedrick extract --in /tmp/log.offset --out /tmp/me.offset --feed "@vYqLJ+S8RSwrgU6Nxja0kM3d19oWqjv9Og2JCbDd8+U=.ed25519" --invert

NICE, thanks for the README @seanb, I feel a lot safer about using this module now!

check it out :

» ls -lah *.offset
-rw-r--r-- 1 mixmix mixmix 667M May 30 19:57 everyone_but_sbot.offset
-rw-r--r-- 1 mixmix mixmix 962M May 30 19:54 log.offset

OK yeah, I'm safely out the other side (I think)

NOTE - if you're going to use this tool, you need to build all your flume indexes. The easiest way to do that is to delete everything in e.g. ~/.ssb/flume EXCEPT ~/.ssb/flume/log.offset as that is your core source of truth which you need to hang on to. If you start up your client it will then rebuild all those indexes (which will take a bunch of time)

Sidenote - this lead me to publish ssb-unread@1.0.8 which now preserves the off-chain state of which messages have been read in this particular edge case.

did you dip into the stream @seanb ? Maybe a nice feature would be something which for people at 3 hops away it gives you a sampler ... like maybe 500 messages, and then you opt into more. ... ore maybe we tap the breaks on any account that's publishing > 200 messages a day ?!

oh god, now it's @/iGURN4KIyJWMUDdbpjaZQHyCL9fjpFWGObGZ8zhzts=.ed25519 ...

@mixmix quick brainstorm before I had my coffee, I might be being naive or stupid here (or both) but what about a bot that agreesively follow people and try to detect those feeds which are feedspaming and produce reports/blocklists/etc. I'm thinking about antibodies for ssb. Hermie needs an imune system.

@alanz I like that idea a lot and agree with you. As I said, my idea might be stupid. Does @mixmix sampler idea affects replication right?

I also like the sampling idea. It may add a bit of cognitive load on the user, but that can be mitigated with some good heuristics like @mix suggests, which could even be made configurable per user.

I think this is interesting. Just like there are lite clients in the blockchain world who don't store the whole history, but only the blocks needed for the current state.We could store even less. But we need then a way of checking with multiple peers to do some sort of validation.