@andrestaltz thanks, I don't know why I missed that. It is a good sign I think that we could have three or more different markdown engines rendering posts in mostly mutually compatible way. I read in the readme for remark-parse
that remark
has a GFM option which defaults to true, and a CommonMark option which defaults to false, but it doesn't mention about the line break handling, nor was I able to find out about that from looking in the code. I'm not able to check right now but I think I recall seeing GFM-style line break rendering in Manyverse.
@christianbundy thanks for being open to this; I do wish to resolve it. And also about RFC 7764 (and 7763), I didn't know about those.
I count 153440 posts today in my local flumelog (so not exactly "hundreds of thousands"). How many were composed with the assumption of Markdown or GFM is hard to say. Many posts may not make use of single-linebreak significance, GFM features, or Markdown at all. To best estimate, I think it would require laboriously making a guess for each feed to pick a post at which it looks like they started using ssb-markdown v4.0.0 or Patchbay v7.15.6, and assume that they used that from that point until now. But for are an upper bound, here are some dates and calculations:
- ssb-markdown v4.0.0 published 2018-11-25
- patchcore depending on ssb-markdown@markdown-it on 2018-11-19, ssb-markdown@^4.0.0 in release v2.1.0 on 2018-11-25
- patchbay v7.15.6 released 2018-12-04
- patchwork using ssb-markdown@^4.0.0 on 2018-11-25, untagged
Counting my local posts received since 2018-11-19, I get 11334; since 2018-12-04, 8544. Of those messages, ones that contain the pattern /[^\n]\n[^\n]/
, which I think should match a solitary newline, number 2286 and 1622, respectively. Source: &oq+ax1i.... Personally I have only noticed a handful of recent messages that appeared to have line breaks incorrectly rendered, but there are probably more I did not notice.
I think it should be possible to be able to render markdown from a message content as its author intended, without user intervention, if the message includes information needed to resolve ambiguities, such as a variant property like you mentioned. Or at least, I think it is a worthwhile goal and probably attainable. Even if such messages are not rendered completely correctly immediately by all clients, just having the needed context in the raw message would mean the clients could be eventually changed to render them consistently.
ssb-markdown
accepts an opts parameter. A client could take a variant
property from the message content and pass the renderer an option based on that. Similarly could probably be done for ssb-marked and marked's functions. When publishing a message, I think it would be okay for the client to use a hard-coded variant according to the developers' judgment. Or the client could take it from config or the user's feed, or the composer UI - but just for consistent message rendering this would not be necessary.
I'm not sure about type: 'text/markdown'
, unless it was a second-level property, or a blob mention, since we already have type: 'post'
as a top-level property which is well-supported. If posts use HTML, I'm not sure that would be simpler. If a client is not targetting a web browser, it would have to parse the HTML. If it was web-based, it would still need to sanitize the HTML, unless its sandbox takes care of that. Also it might have to parse the HTML to rewrite SSB refs to URLs, and to render a message as plain text for inline use. Also, all implementations would need to be changed to accomodate this as a new feature - and they would still probably need to handle old/existing messages.
I think GFM is a standard as is CommonMark, as they were both added to IANA's Markdown Variants registry. SSB Markdown as it is might not be exactly any of these, since it uses SSB-specific features for link rendering. Would it be appropriate to modify the variant to indicate it is SSB-specific, such as SSB-GFM
and SSB-CommonMark
? I think it could be useful to distinguish them from the pure variants until/unless the message text can be handled by a non-SSB markdown implementation, i.e. the message text uses links only with URIs, not sigil-refs/mentions.