> ## Documentation Index
> Fetch the complete documentation index at: https://docs.usebench.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Code fixes

> Use a coding agent to propose a patch, then validate it independently.

<Note>
  Local preview. Bench exports repair evidence and includes a local validation runner.
  Hosted coding-agent execution, automatic PR publication and deployment are not enabled.
</Note>

## The workflow

```text theme={null}
Failed evaluation -> fix brief -> coding agent -> candidate patch
                                                   |
                          regression + incident + holdout checks
                                                   |
                                      validated for review
```

An agent writes the proposed change. Bench's validator determines whether it passes
the configured checks. A validated patch is not a deployed fix.

### 1. Export the evidence

Open an evaluation in **History**, choose **Fixes**, then select **Fix brief**, or call
`bench_get_fix_brief` with the completed run ID through MCP. This is a read-only
operation with no evaluation charge. The brief includes that run's rubric, cases,
baseline and candidate evidence. Deleted or incomplete evidence cannot be exported
as a repair task.

Treat a brief as sensitive source data. Share it only with an approved coding agent.
Saved optimization cases are not independent holdout cases.

For a quicker handoff, choose **Prepare fix prompt** and **Copy prompt**. Paste it into Codex, Claude Code or another trusted coding agent. **Open in Cursor** uses Cursor's local application link. It does not route your source evidence through a web redirect. If the prompt exceeds the link limit, Bench copies the complete prompt and opens a short instruction asking you to paste it. Review it before running the agent. [Cursor documents this handoff](https://cursor.com/docs/reference/deeplinks).

### 2. Pin the source and checks

Confirm a clean checkout matches the evaluated code, then record its full commit SHA.
New runs can include the commit recorded in their source snapshot. Older runs still need an explicit source revision; Bench does not invent one. Choose a small set of editable application files. The local runner rejects
protected tests, policies, dependency manifests, hidden files and credential files.

An operator supplies a validation plan outside the source checkout:

```json theme={null}
{
  "schema_version": 1,
  "source_revision": "REPLACE_WITH_FULL_40_CHARACTER_COMMIT_SHA",
  "allowed_paths": ["src/refunds.py"],
  "checks": [
    {
      "name": "existing_behavior",
      "kind": "regression",
      "argv": ["python3", "/trusted/checks/regressions.py"],
      "timeout_seconds": 60
    },
    {
      "name": "reported_failure",
      "kind": "incident",
      "argv": ["python3", "/trusted/checks/refund_failure.py"],
      "timeout_seconds": 60
    },
    {
      "name": "unseen_cases",
      "kind": "holdout",
      "argv": ["python3", "/trusted/checks/holdout.py"],
      "timeout_seconds": 60
    }
  ]
}
```

Commands execute with the snapshot as their working directory. Use an absolute
interpreter path when needed. The runner does not install dependencies; use a
pinned, pre-provisioned environment. Executable repository file modes are not
preserved, so invoke scripts through their interpreter. Checks must return `0`
for passing assertions, `1` for a reproduced behavior failure, and another exit
code for setup or infrastructure errors. Do not hide provider failures as a failed
quality assertion. Baseline regression checks must pass; each incident check must
fail on baseline and pass on candidate. Candidate holdout checks must pass.

Keep the holdout separate from the brief and authoring workspace. The local runner
does not provide OS isolation or stop a malicious program reading other files.

### 3. Generate the candidate

Give the brief, source files and their SHA-256 hashes to Claude Code, Codex, Cursor,
or an approved agent adapter. Ask for a minimal repair, not changed tests or weaker
criteria. The proposal contract is:

```json theme={null}
{
  "changes": [{
    "path": "src/refunds.py",
    "before_sha256": "SHA256_OF_THE_ORIGINAL_FILE_BYTES",
    "content": "Complete replacement source text, not a partial snippet.\n"
  }]
}
```

Only existing, allowlisted UTF-8 files are supported. File additions, deletions,
symlinks, submodules, dependency upgrades and infrastructure edits need a separate
workflow. The author has no authority to declare validation successful.

### 4. Run validation

From the local `bench-pipeline` checkout:

```sh theme={null}
.venv/bin/python -m pipeline.code_fixes \
  --repo /absolute/path/to/clean/source \
  --plan /absolute/path/to/validation-plan.json \
  --brief /absolute/path/to/bench-fix-42.json \
  --proposal /absolute/path/to/proposal.json \
  --output /absolute/path/to/new-review-directory \
  --allow-local-execution
```

For automatic generation and validation in one invocation, replace `--proposal`
with an operator-owned adapter command:

```sh theme={null}
--agent-command '["/trusted/agent-adapter","--workspace","{workspace}","--brief","{brief}","--output","{proposal}"]'
```

The runner supplies a separate authoring snapshot, brief and output path. The adapter
must write the proposal contract and exit. Bench reconstructs a fresh candidate from
the pinned source, so it ignores direct agent edits outside the proposal. This preview
includes the command contract, not a bundled Codex/Claude API adapter. Configure an
approved provider proxy in the adapter; ambient credentials are not inherited.

<Warning>
  `--allow-local-execution` authorizes real commands on your machine. Use it only with
  operator-owned, trusted repositories and commands. It is not a sandbox. Never expose
  this runner directly to uploaded customer code or execute it inside the API process.
</Warning>

### 5. Review the outcome

The new output directory contains `candidate.patch` and `report.json`. Reports record
the source revision, plan and brief hashes, file hashes, check exits, timeouts and
output hashes. Raw command output is not retained. Output is bounded to 1 MB per
command, checks to 300 seconds each, and an authoring command to 300 seconds.

`validated_for_review` requires every configured gate. Rejected candidates remain
rejected even if the agent says they passed. The original checkout stays unchanged.
The emitted patch is applied to a fresh snapshot and checked against the validated
candidate, including exact file bytes and line endings.
No branch is pushed and no PR is opened by this runner. Review a patch against its
recorded base before applying it. Changes after validation require validation again.

## Hosted rollout

Use a separate isolated worker, such as an E2B microVM, before enabling repairs for
untrusted repositories. Separate generation and validation sandboxes, restrict egress,
keep provider/GitHub credentials behind an authenticated proxy, pin dependencies,
enforce tenant and spend budgets, and destroy sandboxes on every exit path.

The existing `bench_open_prompt_pr` tool writes supplied files to GitHub. It does not
enforce this validator's receipt or revalidate code, so it is not the production
auto-fix publisher yet. A hosted publisher must verify the receipt and current base
again, then require permission before opening a PR. Never automatically merge or
claim a production issue is resolved before rollout measurements confirm it.

See [validation and rollout](/guides/validation) for the release gates.