Get started with Bench MCP

Bench MCP connects your coding agent to Bench. Scan a repository, evaluate a prompt, and review the recommended prompt and model without switching tools.

In the staging preview, MCP connection is included on Free, Builder, Growth, and Enterprise. Production eligibility follows the plan shown in your account until that release is deployed. Evaluations use your account’s allowance. Connecting does not start an evaluation or apply changes. Testing staging? Copy the connection command from its MCP page. The production URL below does not expose unreleased staging tools.

Connect your coding agent

In the Bench app, open MCP, select your client, then Connect Bench. Copy the command or configuration from your own environment. Other clients shows the server configuration for compatible MCP clients. After connecting, copy Paste into your coding agent to discover systems and plan a first bench. The starter prompt asks for approval before using an evaluation or applying a fix.

Claude Code
Cursor
Codex
Other clients

Run this in your terminal:

claude mcp add --transport http bench https://mcp.usebench.ai/mcp

Open Claude Code, run /mcp, and follow the sign-in prompts. Use the same account you use in the Bench app.To inspect the configuration:

claude mcp get bench

The default scope is the current project. Add --scope user if you want the connection available across projects.

Add Bench to your personal ~/.cursor/mcp.json. Merge this entry with any servers you already have:

{
  "mcpServers": {
    "bench": {
      "url": "https://mcp.usebench.ai/mcp"
    }
  }
}

Open Cursor’s MCP settings and complete the authorization prompt. Adding the JSON alone does not confirm that authentication succeeded.

Add the remote server, then complete authorization:

codex mcp add bench --url https://mcp.usebench.ai/mcp
codex mcp login bench

For a local Bench environment or API-key connection, copy the exact command from that environment’s MCP page. Do not send a local key to a production endpoint.

Use the client’s remote MCP connection settings with:

https://mcp.usebench.ai/mcp

The client must support the server’s HTTP transport and authentication. If it cannot complete sign-in, use the API-key setup shown on the MCP page in your Bench app.

Check the connection

Ask your coding agent:

Use Bench to show my account and connected repositories. Do not start an evaluation yet.

The agent can use bench_whoami and bench_list_repos. If the repository you need is missing, connect GitHub in Bench and confirm that the GitHub App has access to it.

Run your first bench

Scan a repository or provide a prompt

Ask: “Scan owner/repository on branch-name. Show the prompts you found and explain which one you would evaluate.”A scan discovers code and prompts. It is not proof of runtime quality. You can also use Bench’s prompt-upload flow when you do not have a connected repository.

Confirm the evaluation

Ask: “Bench this prompt. Explain the input and context first, then ask me before using an evaluation.”Bench creates or reuses criteria and test cases, scores the extracted prompt, and evaluates candidate improvements within your plan’s limits. Optional business context helps define the outcome, rules, and exceptions that matter.

Follow progress

Your agent can poll bench_get_evaluation. Current system-first runs do not require a rubric-review step. An older run may return awaiting_review; inspect its saved cases before using bench_submit_run_review to continue.

Investigate the result

Ask: “Show the failed checks, explain why they failed, and compare the proposed prompt and model with the baseline. Include cost and any unscored checks.”Use the actual saved benchmark and optimization results. A missing result or failed provider call is not evidence that the AI behavior is bad.

Review a proposed fix

Use bench_get_fix_brief with a completed run ID to export its pinned repair evidence. An approved coding agent can propose changes, then an independent validator checks them. See Code fixes. This read-only tool does not start a paid evaluation, execute code or publish a PR. A recommendation is not a deployed fix. Inspect the proposed prompt, selected model, measured quality, cost, and limitations before changing your application. bench_open_prompt_pr is an external write action. Ask explicitly before using it. It requires a connected repository and a supported prompt location. Uploaded prompts do not have a repository file to change. Creating a PR does not merge or deploy it.

Prompt evaluation does not execute every part of a multi-agent application. Tool side effects, orchestration, real-time voice behavior, and production outcomes need runtime evidence and their own execution checks. Do not describe a prompt-level result as a validated end-to-end system fix.

Useful requests

The system-first preview adds system context, test-library and production tools. They require the matching API and MCP deployment.

bench_start_system_evaluation: bench all recognized prompts in a system after confirming allowance use. This is the Free whole-system path. bench_start_evaluation keeps its explicit prompt-selection behavior.
bench_list_systems / bench_get_system: find a system and its prompt IDs.
bench_get_system_context / bench_list_context_sources: inspect versioned evidence.
bench_save_system_context: save an explicit correction for future runs.
bench_list_test_library / bench_save_test_case / bench_save_criterion: customize cases and criteria.
bench_list_datasets / bench_import_dataset_cases: use an existing mapped dataset.
bench_evaluation_allowance: check the account and key limits before spending.
bench_get_evaluation_artifacts: inspect the exact saved run evidence.
bench_list_production_traces / bench_get_production_trace: inspect recorded behavior.
bench_check_production_span / bench_get_production_check / bench_retry_production_check: run and follow a background check.

Read Production checks for capture, consent and retention. Repository-restricted keys use the system-scoped tools above. Legacy per-prompt result tools cannot verify all the repositories behind old evidence and reject restricted keys. A shared prompt does not share another system’s private cases.

“Find which prompts are responsible for this behavior.”
“Explain this failed check and the evidence used to judge it.”
“Compare the recommended model’s cost and quality with the baseline.”
“Prepare a prompt change for my review. Do not open a PR yet.”

Troubleshooting

The client cannot connect

Check the configured URL, inspect the client’s MCP status, and complete sign-in again. For a local environment, copy the configuration from that environment’s MCP page instead of mixing production endpoints with local credentials.

My repository is missing

Confirm the Bench account, GitHub account, and repository access granted to the Bench GitHub App. Scoped API keys may intentionally hide other repositories.

I have no evaluations left

MCP connection is free, but evaluations share your plan allowance. Check Billing for the current balance and plan options.

The evaluation is paused or failed

Read the run’s status and error. A legacy review pause needs a case review. An execution failure needs a retry or configuration correction, not a fabricated quality score.

Read Security and limits before sharing access or enabling write tools.

Start here

Evaluate and improve

MCP

SDK

Development

Get started with Bench MCP

Connect your coding agent

Check the connection

Run your first bench

Review a proposed fix

Useful requests

Troubleshooting

​Connect your coding agent

​Check the connection

​Run your first bench

​Review a proposed fix

​Useful requests

​Troubleshooting

Connect your coding agent

Check the connection

Run your first bench

Review a proposed fix

Useful requests

Troubleshooting