@SoapDog (Macbook Air M1) this is choice. I was anxious reading the first post when you indicate you opened the pandoc can of worms. I didn't know if I was going to be reading a horror or a heroes journey. Had to skim ahead to soothe my nerves

Getting mermaid diagrams working is more important than simply getting this specific diagram to display correctly. Now that mermaid is available, it is easier to convert the protocol guide to markdown. Also, many of the linked specs use mermaid in their own READMEs.

Got the github action working. That diagram is live at https://ssb-docs.soapdog.org/specifications/index.html

More adventures on the documentation repo

@mixmix in this message mentioned that he created an amazing repo with a clickable mermaid diagram showing the relationship between the various SSB specs.

I promptly told him that I was going import that into the documentation repo. I thought it was going to be simple.

After a quick research, I found a Pandoc filter for processing mermaid diagrams. That filter is built with NodeJS, which just adds NodeJS as a dependency for the project (and chromium, more about that later...)

The mermaid filter generates by default a png file and inline it onto the HTML using a data uri. That is good enough for most purposes, but PNGs don't have clickable elements inside it causing us to lose the links inside the diagram.

One can setup the mermaid inside the markdown to output SVG by changing the fenced block from simply mermaid to a more qualified:

{.mermaid format=svg loc=content/specifications/img}

Which not only sets the output format, but also where to place the generated file.

The first issue with that approach was that it generated a <img> in the HTML with a src that pointed at the value from the loc attribute on the original mermaid diagram. This is not the output path for the HTML which means that the link will throw a 404 because there is no content folder on the deployed HTML structure.

To fix that, I simply created a new Lua-filter similar to the one I build for the Table of Contents. It picks all images and look into their source replacing content/ with empty.

function Image(el)
  if FORMAT:match "html5" then
    -- remove references to the content folder
    if el.src:startswith "content/"  then
      el.src = el.src:gsub("content/","")
    end
 end
 return el
end

This almost worked. The svg file loaded correctly but the links were not working. I suspect that CSP plays a role in preventing links from loaded SVGs from working. It is either that or some other browser security issue.

Apparently, links inside a SVG only work for inline SVGs, this means that I had to build another Lua filter to pick all Images, check which ones were SVG files and inline them.

function loadContent(file)
    local f = assert(io.open(file, "rb"))
    local content = f:read("*all")
    f:close()
    return content
end

function Image(el)
  local old = el.src
  if FORMAT:match "html5" then
    -- remove references to the content folder
    if el.src:startswith "content/"  then
      el.src = el.src:gsub("content/","")
    end
    -- if the image is an SVG, inline it because clickevents only work if it is an inline SVG.
    if el.src:endswith ".svg" then
      local svg = loadContent(old)
      return pandoc.RawInline("html5", svg)
    end
  end

  return el
end

Now, that pretty mermaid diagram works as expected with all its links.

The next steps are:

• Get the damn github action to play nice with mermaid. That mermaid filter requires chromium and pupeteer and who knows what else. It was really easy to install on my machine but it is proving a bit difficult in the github action environment.
• Copy all the specs into the documentation repository and update the links to point to them.

I hope people enjoy these kind of posts, I'm doing them in an effort to highlight that documentation works — when in an early phase; building workflows and pipelines — may involve a lot of little gotchas and tricky-to-fix annoyances.

It takes a long time to figure out such problems and create working fixes. All this is done in hope that the repository will be useful to the community and easy to contribute. There is no need for anyone else to deal with these gotchas, people can simply drop markdown files inside the content folder (and maybe link them in the correct configuration.toml for the folder).