AI Testing Agent: Agents, Add-ons, Prompts

Use an AI testing agent to write the routine tests, run the suite, read failures and drive a browser through your main flows, while a person decides what must be covered and checks what each test actually asserts. Good picks are a terminal coding agent, a read-only or test-only subagent and a browser add-on such as Playwright MCP. Model cost per test is cents. Checked 31 October 2026.

The job in one table

What the job is

Testing answers one question: does the software do what we meant? It is a good job for agents because the world gives clear feedback. A test passes or fails, and a browser flow reaches a page or does not. Anthropic's guide to building with models notes that agents do well where the environment provides ground truth at each step, such as tool results or code execution. Tests are exactly that signal.

The catch is the one every tester knows. A test can pass and prove nothing. An agent that is told to "make the tests pass" may weaken the assertions until they do. So the human job moves from writing every test to reviewing what the tests claim.

Which agent fits

For picking between these tools, see AI coding agents and choose an AI coding agent.

Add-ons and kit items that help

A worked setup that combines the first and third rows is in Claude Code with Playwright for browser QA. Prompts for testing sit in the prompt library. Put your test conventions in a CLAUDE.md or AGENTS.md so every run follows them: where tests live, how to run one, which framework, what never to mock.

A safe workflow, step by step

1. Write down what must work. Five user flows and the ten functions that matter most.
2. Give the agent the test conventions in an instruction file.
3. Generate tests for one module at a time, not the whole repo.
4. Run them and read the failures. A failing new test may be a real bug; ask the agent for evidence before you fix anything.
5. Mutate to check. Break the code on purpose and confirm a test fails. A test that still passes is not testing.
6. Review assertions. Each test should assert a result a user cares about, not just "it did not throw."
7. Add browser flows last, against a test environment with test data.
8. Run in CI. A test that only runs on the agent's machine does not protect you.
9. Keep a human owner of the suite.

Rule 5 is the one to insist on. If you have no time for anything else, break one line of code per generated test file and see if the suite notices.

What a good AI-written test looks like

Compare two tests for the same function, a discount calculator, as an example.

Ask the agent for the strong kind in your instruction file: "each test asserts a specific expected value taken from the requirements; include boundary cases; do not mock the unit under test." Then run the mutation spot-check: change the threshold from 100 to 101 and confirm a test fails.

Browser flows worth testing first

A small set of flows protects most of the value. Pick the ones where a failure costs you customers or money.

• Sign up and sign in.
• The main thing a user comes to do, from start to finish.
• Checkout or the equivalent paid action, using a test mode.
• Password reset.
• A form that creates a record, then viewing it.

For each flow write the steps and the expected result in plain words first, and let the agent turn that into a script. Run the flow at a phone width as well as on desktop, since layout bugs often show at the narrow size. Keep browser tests few and stable; unit tests are cheaper and faster, and a pyramid with many unit tests and a few browser flows is easier to maintain than the reverse.

Triage: reading a red build

When the suite fails, an agent can save the first twenty minutes. Ask it to answer four questions in order: which tests failed, what changed in the last commit, whether the failure reproduces locally, and whether the cause is in the test or in the code. Require evidence for each answer, such as a log line or a diff hunk. A model that says "probably a race condition" with no evidence has told you nothing. The bug triage prompt gives a layout for turning the answer into a ticket.

What it costs

Model. From Anthropic's pricing page, checked 31 October 2026: Sonnet 5.5 is $2 per million input tokens and $10 per million output; Haiku 5.5 is $0.10 and $0.50.

Worked example (the token counts are our assumptions). Generating tests for one module: 8,000 input tokens (the code and examples) and 3,000 output tokens:

An agent that also runs the suite, reads failures and retries uses several times that. Browser agents read page snapshots on every step, which costs many more input tokens than code alone. We have no figure for tokens per snapshot, so read your own usage log.

Seats. Terminal agents start at about $20 a month (Claude Code Pro $20, $17 annual; Codex Plus $20; Copilot Pro $10; Cursor Plus $8). See AI coding agent pricing compared. Playwright MCP is free; you pay for the model that drives it. Its README also lists opt-in capability groups you switch on with the --caps option, including --caps=testing for test assertions and --caps=network for request routing, so an agent only gets the extra tools you enable.

People. The review step is the cost that matters. A hundred generated tests at two minutes of review each is over three hours, which is still far less than writing them.

Risks and how to review them

Playwright MCP gives an agent control of a real browser, so treat it like handing over your keyboard on that machine. Our add-on page covers its limits and risks. The MCP spec says tools represent arbitrary code execution and hosts should get user consent, and Claude Code's docs say to verify you trust each server.

Measure it

Coverage percentage is a weak target: an agent can raise it with tests that assert nothing.

Common mistakes

• Generating tests for the whole repository at once. Review becomes impossible; go module by module.
• Letting the agent change production code to make a test pass. Restrict its edit rights.
• Counting tests, not checking them. Ten strong tests beat a hundred empty ones.
• Testing against production. Use a test environment and throwaway data.
• No CI. Tests that only run when someone remembers do not protect a release.
• Skipping the human. Someone must own the suite, read the failures and decide what to fix.

A first afternoon

Pick one module that changes often and has few tests. Ask the agent to list the behaviours it can see in the code, and compare that list with what you think the module should do. The differences are your first findings. Generate tests for the behaviours you agree on, run them and break one line on purpose to confirm the suite notices. Write down how long it took and how many generated tests you kept. That one afternoon gives you a real number for your own code, which is worth more than any vendor claim.

Who should do what

Related tasks: AI code review for the review of the change itself, AI documentation for keeping docs true to the tests and AI tools for business for choosing across jobs.

Our take

Testing suits agents because tests give honest feedback, and it fails when nobody checks what the tests mean. Constrain the agent to test files, insist on a quick mutation check and keep a person on the release call. We built this from vendor documentation, our add-on pages and published guidance, and have not run these agents on a production suite ourselves, so pilot on one module before you trust the pattern.