Ok, the string I pasted into my message just contains the replacement character U+FFFD
, not any invalid utf8, so that's a plus, my feed won't be broken.
Turns out Buf.toString
doesn't check validity at all, it even returns a string for Buf.from([255], 'utf8').toString('utf8)
. Which is technically allowed, since js strings are not required to be valid unicode.
The mismatch happened because I used Buf.toString
for my test data generation, I'll need too look at what exactly the js ssb implementation does. Maybe this is actually handled correctly and all of this was just a false alarm.