ssb-split-publish

You know that thing where you try to publish a message and hit the 8kb limit? I've built a fix (of sorts) for that - publish with this tool and if you hit that limit it will split your message over as many messages as it takes. (you just have to define how to split!)

https://gitlab.com/ahau/ssb-split-publish

Here's an example which will crudely split oversized post messages at 4000 characters. It also has an option included which gets the "branch" on the message right so it will be tangled correctly:

const SplitPublish = require('ssb-split-publish')

const splitPublish =  SplitPublish(ssb, splitter, { afterPublish })

function splitter (content) {
  const first = {
    type: 'post',
    text: content.text.slice(0, 4000), // janky!
    root: content.root || null
  }
  if (content.branch) first.branch = content.branch

  const second = {
    type: 'post',
    text: content.text.slice(4000), // janky!
    root: content.root,
    branch: ['TODO']
  }

  return [first, second]
}
function afterPublish (firstMsg, secondChunk) {
  secondChunk.root = firstMsg.value.content.root || firstMsg.key
  secondChunk.branch = [firstMsg.key]
  return secondChunk
}

const content = {
  type: 'post',
  text: 'You know what really grings my gears? {....}', // TOO LONG
  root: null
}

splitPublish(content, (err, msgs) => {
  console.log(msgs)
  // => [
  //   msg1,
  //   msg2...
  // ]
})

This work will soon be part of ssb-crut , which is all transformation based - a content type which does not care at all about being split across messages!

cc @staltz @SoapDog @SoapDog (Macbook Air)

@mixmix @mix.desktop,

I like it but, isn't that the case a single message with an attachment blob? That's how blog messages work, and the beauty of it is that they're a single message whose actual content can be larger than 8k.

I see the other message where you mention this will be a part of ssb-crut, I think is pretty good.

refactor to make that function easier to read:

function splitter (content) {
  return [
    { ...content, text: content.text.slice(0, 4000) }, // first
    { ...content, text: content.text.slice(4000) } // second
  ]
}