Skip to main content

​
Copy this into your coding agent

Paste into Cursor, Codex, Claude Code, or another coding agent in your project.

​
Start with a failed case

Open History, choose a bench and select View details → Fixes. Compare the original prompt with the proposed prompt or model, and read why Bench recommends the change. Review quality and cost together. Failures also surface outside History. The Production and Real app testing tabs open with an attention strip that lists failures first, then proposed fixes. A failed production check starts an automatic re-bench with the failing interactions as new cases (on by default; the owner can turn it off), and that bench appears in Results and History like any other. Each model-heavy stage of a bench may take up to one hour, so a re-bench is not immediate.

​
Share the evidence with your coding agent

Choose Prepare fix prompt, then Copy prompt. The prompt includes the failed case, the expected behavior and the evaluated code revision. You can also download Fix brief, or ask your connected coding agent to use bench_get_fix_brief. Reading a fix brief does not use an evaluation. In Real app testing, a test or case opens in a right-hand drawer with Bench outcome, Output and Case tabs and its own fix prompt for a coding agent. In Production, the drawer for a failed span shows the recorded error message and the tools and models on its timeline, so the agent can start from the actual failure. A coding agent can investigate prompt instructions, model configuration, tool code or workflow logic. A failed test identifies behavior to investigate; it does not by itself prove which part caused the failure. Share briefs only with tools you approve: they can contain your source code and evaluation data. Review the proposed changes before running or applying them.

​
Test the change

  1. Reproduce the original failure.
  2. Apply the proposed change to a test branch.
  3. Run the same case again, keeping its expected result and criteria unchanged.
  4. Run previously passing cases to check for regressions.
  5. Use independent cases that were not used to create the fix.
Real app testing check the actual tool calls and final outcome. A higher prompt score alone does not prove that a tool or workflow works.

​
Review before rollout

If you create a pull request through Bench or MCP, review its repository, branch, files and test evidence. Creating a pull request does not merge or deploy it. After deployment, use production evidence to check that the original problem is resolved for real users. Continue with validation and production checks.