A11yAudit is a self-hosted n8n automation that screens digital health content - patient portals, health information pages, discharge instructions - for accessibility barriers and comprehension problems.
You submit a URL or paste text. It returns a prioritised list of barriers with plain-language explanations and concrete fixes, a set of separate scores, a draft accessibility statement, and a record of which rules fired and why, stored in a Postgres database.
Problem Statement
Health content must be accessible under the European Accessibility Act and the German BFSG. That's a real compliance deadline for hospitals, insurers, and public health publishers, not a nice-to-have. It also has to be understandable by people reading it while in pain, frightened, or medicated, with reduced cognitive capacity exactly when comprehension matters most. Existing tools solve half the problem each. Accessibility checkers test markup but reduce language to a syllable-counting grade; they can't tell "Take 1 tablet BD" is a dosing instruction nobody explained. Health-literacy instruments like PEMAT-P and the CDC Clear Communication Index catch that, but they're scored by hand, too slow to run at the scale a compliance deadline actually demands.
How it works
The AI proposes. Deterministic rules dispose.
It's a self-hosted n8n pipeline. Nine deterministic WCAG checks run first, no AI involved. A safety prescreen flags dosing, emergency, and risk language via regex before anything else happens. One AI call, temperature 0, validated against a strict JSON schema, proposes PEMAT-P/CCI verdicts and additional findings. Each one gets checked against the literal source text before it's trusted, and a finding is discarded if its quoted evidence doesn't appear verbatim on the page. Nine deterministic rules decide whether a case needs human review, and they still fire correctly if the AI call fails entirely. A full audit (markup, language, safety terms, and a validated AI pass) completes in 59 to 69 seconds, measured over nine real runs, not estimated.
Tech stack
n8n (self-hosted) - orchestrates the whole pipeline, node by node, no black-box agent framework in between.
PostgreSQL - audits, findings, instrument verdicts, and the review queue; a least-privilege role for the pipeline's own writes, separate from n8n's internal database.
Claude (Anthropic API) - the single validated AI call per audit, temperature 0, schema-constrained output.
Docker Compose -for local and CI infrastructure
GitHub Actions - for the six-job CI pipeline, including the end-to-end n8n run.
Stress-tested and reproduced independently
Two harnesses simulate the worst case instead of the average one. One replays a prompt-injection attempt embedded in the audited page's own content. The other simulates the AI reporting a perfect result: zero findings, every item marked "pass," just to see whether the deterministic safety net still catches it. It does. Human review still gets triggered no matter what the AI reports, because the routing decision never depends on the AI being right.
The headline result, a screening score rising from 52 to 100 on the same page after its own fixes were applied, was reproduced from a clean install on separate hardware weeks after the original measurement, starting from an empty database. The evidence-verification mechanism discarded roughly a fifth of the AI's own proposed findings in that run instead of silently keeping them.
Under the hood
Running 24 real external health pages (English and German) through the published pipeline during testing surfaced real defects that a synthetic fixture never would have: navigation text extracted as article content, a SQL comment that n8n evaluates as a live expression even inside a crash on a quoted HTML id attribute. CI runs six jobs on every push, including one that boots a real headless n8n instance, imports the real workflow, submits a real form, and checks the resulting database row.
The project audits itself, too
The decision log records every design choice on this project, 98 entries in all went stale for eleven entries once. They existed only as one-line changelog summaries for weeks, and the gap was found only because someone thought to check again, not by anything automatic. It's documented as a real process failure rather than hidden or quietly backfilled, and it led to four deterministic checks, wired into CI and a local pre-commit hook, that now catch the exact kind of drift that caused it without anyone needing to ask. The project applies the same evidence discipline to its own documentation that it applies to AI output: a "fixed" claim gets re-checked, not trusted.
What it is not
Stated plainly, because these limits are part of the design rather than gaps in it:
It produces a report, not accessible content. A human confirms the findings, and a content owner rewrites the page. Both steps are outside this system.
It screens a listed subset of WCAG 2.2. Colour contrast, keyboard operation, and anything rendered by JavaScript are out of scope and declared in every report. No conformance claim is made.
The instrument scores are an unvalidated adaptation. PEMAT and the CDC Index were built for trained human raters. Applying a subset of their items via an LLM is labelled accordingly and never presented as an official score.
Accuracy has one small data point, not a validated measurement: 74.5% (79.5% adjusted), one rater, two fixtures, not PEMAT's own two-independent-rater design.
Per-call AI cost isn't tracked, and the intake form has no authentication. Both remain open gaps.
