QA Buddy turns an API specification into test cases a QA engineer can read, correct and approve, and then into real test code written the way that engineer's own team writes it. The engineer decides at every step. The AI only proposes.
You hand it an OpenAPI or Swagger file, or a link to one, and it comes back with test cases as plain-English cards: what to do, what should happen, which endpoint it covers. You edit the cards by hand or ask a copilot to add, tighten or remove cases, and both routes change the same list. Once approved, you export them as Markdown or CSV, or take one more step: point QA Buddy at your test-automation repository and it writes Java tests that match how your team already works, using your base classes, your naming and your existing examples. Generated code passes seven fixed checks and your repository's own compiler before you see it, and it is never executed. You review it on a code screen with an assistant that can explain, change or recompile a file, and the result leaves as a zip, a patch, or a pull request on its own branch. It never touches your default branch, and nothing is sent to a model without your consent to an exact list of files.
I built it in Python with FastAPI and a React front end, LangChain and LangGraph for the AI parts, SQLite for anything that must survive a restart, and OpenRouter so every model role can be swapped and measured. The generation logic lives in a core that knows nothing about the web, the database or the login system, and a lint rule in CI keeps it that way; the original Streamlit shell was replaced by React without changing a line of it. Alongside the product is a standing evaluation. Every AI surface has a hermetic tier that runs in CI and a live tier whose scorecards are committed, under the rule that no single day's run may set a pass bar. Where it fails, it says so with numbers: the case copilot follows instructions in its own vocabulary every time, but failed 10 of 41 written the way people actually type. I found that three days before the freeze and shipped it published and unfixed, rather than patch a prompt on the day its numbers needed to be stable.
The lesson that changed how I work came from status codes. Some generated cases arrived without an expected HTTP status, so I added a one-line hint to the prompt telling the model which status to expect. Measured at temperature zero, that hint killed a whole batch of cases in six runs out of six, and taught the model to leave the status out wherever I had not supplied one. Stronger wording fixed neither problem. Making the field required made the model invent codes the spec never declared. What worked was to stop talking to the model about it at all: derive the expected status from the spec in plain code, and fill the gaps afterwards with a visible note saying what was restored and why. Completeness on the worst spec nearly doubled and nothing else moved. I now treat every fact that code can derive as code's job, keep the model for judgment, and check that rule by measurement rather than by whether the prompt reads well.
