Copy this into your coding agent
Paste into Cursor, Codex, Claude Code, or another coding agent in your project.Beta release. All four languages support tracing, real app tests and scripted simulations.
Install
Package on npm · Source and examplesRecord one interaction
Run the installation command above in your server project. Create a repository-scoped key on the SDK page in Bench and keep it in your server’s secret store.NEXT_PUBLIC_ or VITE_ for the key. Set BENCH_API_BASE_URL to the endpoint shown in your account.
Use the API address and installation command shown on the SDK page in your Bench account.
The example uses TypeScript. In JavaScript, remove the ! non-null assertions
and validate required environment variables when your server starts.
Let the framework report the structure
If the application already emits OpenTelemetry spans (Vercel AI SDKexperimental_telemetry, OpenAI Agents, LangChain), register Bench’s exporter on
its tracer provider instead of wrapping calls. No OpenTelemetry dependency is
added; the exporter is structurally typed.
bench.recordExternalSpan({ traceId, spanId, parentSpanId, name, startedAt, endedAt, attributes }), keeping the framework’s identifiers; the kind is
inferred from the gen_ai.* attributes (gen_ai.operation.name first, then
gen_ai.agent.name, gen_ai.tool.name, gen_ai.request.model).
Failed spans exported through the bridge keep error.message (the span’s status
description) and exception.type / exception.message, never the stack trace.
Bench shows that message in the span’s drawer under Production. When a span
does not report bench.cost.usd, Bench estimates its cost from the recorded
tokens and labels the estimate ~.
Register the model calls
Traces never carry the editable template. After the first request, register the model calls the system makes from the repository, one entry per call, using theregister_calls operation (POST /api/ai-systems/{id}/calls) or the
bench_register_calls MCP tool, then put the returned component ids on the
matching model-call spans with componentId. Your coding agent does this for you
when set up from Bench’s setup prompt.
Wrap agents and tools
Replace the synthetic callback with an existing application operation. Nest tool calls withkind: 'TOOL'; awaited child calls keep their parent trace. Wrap the
complete invocation for Deep Agents, LangGraph or custom server code when the
framework does not emit OpenTelemetry spans. These are manual wrappers; prefer
the exporter above where the framework already reports its structure.
For streams, keep the span open until the stream is fully consumed. Fire-and-forget
work may finish after the request and needs its own lifecycle handling.
captureContent defaults to false. When enabled, pass input and let the callback
return the output. Built-in filters run before sending. Add a redact(value)
callback for extra application fields. Read privacy and redaction.
Call await bench.flush() at serverless request boundaries. Reuse the client for
long-running servers and call await bench.shutdown() after active work finishes.
bench.stats and onError expose delivery failures without changing app results.
Set componentId to a real prompt component from Bench when linking events to
criteria; it is sent as bench.component_id. Never invent IDs. SDK setup keys
cannot spend evaluation credits. Report bench.cost.usd when your provider gives
you the price; otherwise Bench estimates cost from tokens and labels it ~.
Test application behavior
This client includesevaluateSystem for individual cases and simulateSystem
for scripted customer conversations through your app. Both require your actual
application function and test dependencies. Read real app testing
for complete examples, then production checks to evaluate
recorded interactions.
See SDK setup for connection status, environments and troubleshooting.