Audit: mermaid diagrams linted in CI¶
What we check¶
A repository whose markdown contains any mermaid fence carries
tools/mermaid-lint.sh and runs it from a CI workflow.
Mermaid fails at render time, not at commit time. A diagram with a syntax error commits cleanly, passes actionlint, shellcheck, flake8 and skillsaw alike, and then shows an error box on GitHub and nothing at all on the mkdocs sites. Nothing else in CI reads a diagram, so until this check existed a broken diagram was found by whoever next looked at the page.
Both halves are required. The script alone is a thing nobody runs, and a workflow step alone would mean each project inventing its own invocation of a container -- the point of shipping the wrapper is that the docker arguments, the entrypoint override and the exit-status handling are written once.
A diagram is recognised by a ```mermaid fence: backticks,
with no space before the language. That is deliberately narrower than
markdown allows, because it is what mmdc recognises -- a
~~~mermaid block renders nothing and exits zero, so counting one
would mark a repository covered for a diagram its linter never sees.
The script draws the same line, and then goes further than the audit
can: where the audit merely declines to count a fence mmdc cannot
read, tools/mermaid-lint.sh refuses it. There are four such fences
-- a tilde-fenced block, one written as ``` mermaid with a
space before the language, one opened with four or more backticks, and
one carrying anything after the language, as in
```mermaid title=x. GitHub renders all four as diagrams even
though mmdc reads nothing in any of them, so failing open would ship
an unlinted diagram through the exact gap the linter exists to close.
The first two failed open as a skip: the run printed "nothing to
lint" and exited zero. The last two were worse and were found by the
automated reviewer on instar#545 rather than by design -- the file was
selected, sent to the renderer, found to contain no chart, and
reported ok inside the "Linting N file(s)" count. A diagram nobody
rendered, wearing the shape of one that rendered cleanly.
Instead the script names the file, the line and what to change and exits 1, alongside any parse errors from the same run; a refusal outranks the renderer's status, so a broken diagram is never reported under a failed image pull's 125. A fence with more than one fault is told the target form outright, since naming the first correction only sends the author round again.
The audit's regex and the script disagree about two of the four, and
in the safe direction both times. `mermaid` does not match
`MERMAID_FENCE_RE`, so a repository whose only diagram is written that
way is N/A here and red in the lane -- the tilde direction again. ``mermaid title=x ```` does match, so such a repository is
counted as having diagrams, told it needs the lane, and then told by
the lane to fix the fence. Neither outcome calls a repository covered
for a diagram nothing renders, which is the property that matters.MermaidLintScriptTestinscripts/tests/` pins that behaviour, and
pins the audit's narrower answer next to it.
The two therefore give deliberately different answers to the same input, and a repository whose only diagrams are tilde-fenced sees both: N/A here, red in the lane. That is the intended direction. The reason the audit does not count such a block was that counting one would call the repository covered for a diagram nothing renders; refusing the block removes the diagram rather than the coverage.
The script classifies fences by tracking fence state rather than by matching lines, so a fence shown inside a longer fence is an example rather than a diagram -- otherwise a page documenting this rule would fail the repository that wrote it. Nesting is the only way to quote a fence: indented code blocks are deliberately not modelled, because four spaces before a fence is far more often a diagram inside a list item, which must still be linted, than a diagram being quoted. Prose is safe by a separate rule -- a backtick fence's info string may not contain a backtick, so a line opening with an inline code span is not a fence -- without which such a line would open one that never closes and hide every diagram below it.
Blockquotes are the one place the two halves agree and both are
wrong. Neither the regex nor the script looks past a leading >, so
a diagram inside a blockquote is N/A here and skipped rather than
refused there, while GitHub renders it and mmdc reports "No mermaid
charts found" for the same file -- measured, not assumed. It is
therefore the fail-open shape the tilde and spaced refusals exist to
close, left open deliberately: because the two halves agree, no
repository is called covered for a diagram nothing renders, and
refusing a blockquoted fence means deciding what a fence nested
inside a blockquoted fence is, which is a rule with its own blind
spot. Blockquoted diagrams are vanishingly rare; a repository that
grows one should promote it to the top level rather than wait for
this to be modelled.
The audit's regex is a line match and has no notion of nesting at
all, which is a divergence with no consequence: a repository whose
only ```mermaid fence is a nested example is asked for a
linter that then finds nothing to lint, and passes.
Repositories with no mermaid diagrams are N/A. This is a check on
diagrams that exist, not a requirement that every project have some;
diagram-format is what moves a repository from hand-drawn diagrams
into this check's scope.
Why a container¶
mmdc renders through puppeteer and so needs a browser. Running it
from the upstream image keeps chromium off the runners, and renders
exactly what the sites will render. It kept node off them too, until
the fleet took nodejs and npm as base packages; that half of the
argument has lapsed.
A DOM-free checker is still ruled out: mermaid's own parse() under
plain node throws DOMPurify.addHook is not a function for
flowchart and stateDiagram-v2, the two most common types in this
fleet, so it reports false failures on exactly the diagrams that
matter -- a question of needing a DOM at all rather than of which
node supplies one. Supplying one with jsdom was excluded because
jsdom pulls in an undici newer than the runners' node, and the
runners now carry node 20, so that parse-only path is untested rather
than closed. Nobody has measured it, and the decision does not change
today.
The cost is smaller than it looks. The image is cached after its first pull, and rendering is about 1.4 seconds per file amortised inside a single container -- ryll's seven diagram-bearing files in ten seconds, ryll and kerbside together in twenty-two. Nearly all of the real cost is the virtual machine the job needs, which is why the shipped workflow is path-filtered to markdown.
The container runs with --network none: rendering is local, and
this is third-party code driving a browser over repository content on
a runner with a docker daemon. Loopback survives, which is all
chromium and mmdc need; a diagram reaching for a remote font fails
loudly instead of rendering differently on a runner with a different
egress path.
What the lane runs on¶
pull_request and push to main or develop, both filtered to
markdown plus the script and workflow themselves. The push trigger is
there because the lane is advisory: a commit that reaches the default
branch without a pull request would otherwise never be linted, and
would surface later as a failure on somebody else's markdown change.
REVIEWS.md is excluded from that filter, and mermaid-lint.sh
excludes the same file from its own tree walk. The two have to move
together. The script lints the whole tree rather than the changed
files, so a file the workflow never triggers on but the script still
reads is the worst of both: a broken diagram merges green on the pull
request that introduced it, then fails whichever unrelated markdown
pull request comes next -- and every developer's pre-push audit --
naming a file that author never touched. MermaidLintDeploymentTest
asserts the two exclusions agree, and that the two triggers filter on
the same paths.
A repository with a tree of its own to leave alone -- a machine-synced
import of somebody else's documentation is the case this exists for --
names it in tools/mermaid-lint-exclude, one path per line, each
turned into an :(exclude,literal) pathspec so that a line is the
path it reads as rather than a glob or pathspec magic. The script
reads that file and the workflow cannot, so the same move-together
rule applies: a line added there is added to both of the workflow's
paths lists as !path/**. A line that drops no tracked markdown
file fails the run, and so does an exclude file that is present but
not committed, because an exclusion that silently drops nothing
leaves the tree linted while the file says it is not.
The runner¶
[self-hosted, vm, debian-12-docker, s], not static. Static runners
have no docker daemon. The label must also appear in
.github/actionlint.yaml, or actionlint fails on the workflow.
Template¶
Template: templates/mermaid-lint/
See: templates/mermaid-lint/README.md
Both files copy directly with no per-project substitution. A project that already has a CI gate job may prefer to add the script as a step there instead of taking the shipped workflow; the template's README covers that, and why a path-filtered workflow must not simply be made a required status check.
Projects¶
Per-project compliance for this criterion is regenerated every morning by the consistency audit: see the compliance page.