Copy this into your coding agent
Paste into Cursor, Codex, Claude Code, or another coding agent in your project.The Python SDK is a preview. Use the installation source provided in Bench.
The planned package name is
trybench-sdk, with the import bench_sdk.Record one interaction
Choose Python on the SDK setup page. Create a setup key and use its installation command. Python 3.10 or later is required.Wrap your application
Usewith bench.trace(...) around a Deep Agents, LangGraph, LangChain or custom
application call. It also works around await agent.ainvoke(...). Wrap tools
inside the same request with kind="TOOL"; nested async tasks inherit their
parent trace. Keep streaming spans open until the stream finishes.
Inputs and outputs are omitted by default. With content capture enabled, pass
input=... and call span.set_output(result). Built-in filtering runs before
delivery. Add a redact(value) callback for your application’s extra fields.
See privacy and redaction.
Call bench.flush() at request boundaries, or await bench.aflush() in async
code. Call shutdown() after active requests finish. Delivery retries once;
bench.stats shows queued and dropped spans. A delivery failure does not change
your application’s result or exception.
Set component_id to a real prompt component from Bench to connect the event to
its saved criteria. Production checks require recorded
content and a key with evaluation allowance.
This package provides tracing. Automatic framework adapters and native application
evaluation helpers are coming soon. The JavaScript application-test helpers
are a separate API.

