Skip to content
Turing College Logo
Submit projectSign In

Usability Testing Suite: evidence-based usability testing for AI products

Usability Testing Suite: evidence-based usability testing for AI products

About this project

Describe a task in plain steps. The suite runs it on your live site, and every finding comes with evidence.

About this project

AI-powered products are hard to test. Answers vary between runs, the assistant asks follow-up questions, and it can be uncertain or wrong. Scripted tests break on this, and a plain LLM review produces confident opinions without evidence.

The Usability Testing Suite gives designers and product teams structured usability and accessibility feedback while they build. The rule behind it: the model proposes, code or a human decides.

What it does

  • Plain-language scenarios. Steps like "enter a request", "click Continue" or "look at the result". Chains link several scenarios into one session.

  • Checked execution. A real browser performs each step in a fixed loop, and code checks that it worked (address changed, text in the field). A failed step stops the run instead of producing findings on the wrong page.

  • Accessibility and navigation, measured. An axe-core scan per step (WCAG) and a navigation-consistency check, both without a model.

  • Guidelines that fit the product. Nielsen's heuristics for every site; for AI products also the matching Microsoft HAX and Google PAIR guidelines, retrieved per segment from a curated catalog.

  • Evidence-bound findings. The evaluator quotes the page, code checks the quote against that step, and a pure function sorts each criterion into Finding, Worth checking, Passed, Looks fine — not verified or Not testable.

  • Built for teams. Issues tracked across runs with history and comments. Roles for admins, designers and stakeholders. Stakeholder feedback is judged against the same criteria. Cost shown per run.

Built for trust

  • Nothing degrades silently. Unconfirmed evidence is downgraded with a visible note.

  • Marketing claims and illustrations never count as evidence of how the interface behaves.

  • A security guard checks page content for prompt injection. Text you enter in a test never appears in reports.

  • Runs in a hardened container: egress filter against SSRF, sandboxed Chromium.

  • About 2,080 unit tests, deterministic click tests at desktop and mobile widths, and a labelled evaluation set with hard gates for the judge.

Results

  • An 8-step chain on a live AI product takes about 2 minutes and costs about USD 0.21.

  • On an AI sourcing platform, the suite flagged a navigation change between start page and task flow. Its screenshots also captured a state bug that the evaluator itself missed: after "Cancel", discarded text stayed in the next input field. That case now drives the roadmap: measure field state instead of relying on page text.

Tech

Next.js 16 · React 19 · TypeScript · Stagehand 4.1 (Chromium) · Mistral Medium 3.5 · OpenRouter · Mistral moderation · axe-core · PostgreSQL 16 · Better Auth · Docker · Caddy · Node test runner

Try it: whenni.henning-concept.de (access on request - Discord-DM or post@henning-concept.de

Created byWoldemar Henning
Published atNovember 17, 2026
CourseAI engineering
Looking for
Freelance projectsNew job
OpenOpen LinkedIn