Skip to main content

Copy this into your coding agent

Paste into Cursor, Codex, Claude Code, or another coding agent in your project.
The Python SDK is a preview. Use the installation source provided in Bench. The planned package name is trybench-sdk, with the import bench_sdk.

Record one interaction

Choose Python on the SDK setup page. Create a setup key and use its installation command. Python 3.10 or later is required.
Open the system’s Production tab. Setup changes from Waiting for first event to SDK events received after the event is accepted.

Wrap your application

Use with bench.trace(...) around a Deep Agents, LangGraph, LangChain or custom application call. It also works around await agent.ainvoke(...). Wrap tools inside the same request with kind="TOOL"; nested async tasks inherit their parent trace. Keep streaming spans open until the stream finishes. Inputs and outputs are omitted by default. With content capture enabled, pass input=... and call span.set_output(result). Built-in filtering runs before delivery. Add a redact(value) callback for your application’s extra fields. See privacy and redaction. Call bench.flush() at request boundaries, or await bench.aflush() in async code. Call shutdown() after active requests finish. Delivery retries once; bench.stats shows queued and dropped spans. A delivery failure does not change your application’s result or exception. Set component_id to a real prompt component from Bench to connect the event to its saved criteria. Production checks require recorded content and a key with evaluation allowance. This package provides tracing. Automatic framework adapters and native application evaluation helpers are coming soon. The JavaScript application-test helpers are a separate API.

Run in CI

Use Bench with your existing pull-request checks. See Run tests in CI for setup, report handling and the features available in each language.