Copy this into your coding agent
Paste into Cursor, Codex, Claude Code, or another coding agent in your project.Beta release. All four languages support tracing, real app tests and scripted simulations.
Install
Package and API reference · Source and examplesRecord one interaction
Choose Go on the SDK setup page. Go 1.22 or later is required. This client uses the standard library.Forward spans your framework already records
If the application already produces spans (its own tracing hooks or an OpenTelemetry pipeline), forward each finished span withclient.RecordExternalSpan, keeping the framework’s trace, span and parent
identifiers, name, start and end times and attributes. Bench infers the kind from
the gen_ai.* attributes (gen_ai.operation.name first, then
gen_ai.agent.name, gen_ai.tool.name, gen_ai.request.model) and draws the
agents, model calls, tools and hand-offs from one real request. Failed spans keep
error.message and exception.type / exception.message, never the stack trace.
Register the model calls
Traces never carry the editable template. After the first request, register the model calls the system makes from the repository, one entry per call, using theregister_calls operation (POST /api/ai-systems/{id}/calls) or the
bench_register_calls MCP tool, then put the returned component ids on the
matching model-call spans with ComponentID. Your coding agent does this for you
when set up from Bench’s setup prompt.
Wrap your application
Replace the callback with your request handler. Pass itsctx into nested
bench.Trace calls for model and tool operations, including goroutines.
Use Kind: "TOOL" for tools. The original result and error are returned unchanged.
For streams, use StartSpan, keep the span open, then call SetOutput or
SetError and End when the stream finishes.
Explicit wrappers work with Google ADK and custom Go applications. Framework
versions may require a newer Go version than the Bench client.
Content capture defaults to off. To enable it, set CaptureContent: true and
provide SpanInput.Input. Built-in filters run before delivery; add a Redact
callback for extra fields. See privacy and redaction.
Call Flush(ctx) at request boundaries and Shutdown(ctx) after active requests
finish. If the request was canceled, use a fresh bounded context for flushing.
Stats() reports dropped events; delivery errors do not replace application errors.
Set ComponentID to a real Bench prompt component to link events to its criteria;
it is sent as bench.component_id. Production checks
work with accepted, linked events. Report bench.cost.usd when your provider
gives you the price; otherwise Bench estimates cost from tokens and labels it ~.
Test your application
client.EvaluateSystem(ctx, options) calls your real application with pinned JSON
cases and returns a redacted report without uploading it.
bench.Expect(value) to assert an output/state, including explicit JSON null
with bench.Expect(nil). Omitted pointers mean no assertion. For state checks,
set ExpectedState and Observe func(context.Context, string) (any, error).
It must read the authoritative test state independently of the final reply.
JSON input numbers arrive as json.Number to preserve large identifiers. Pass the
provided context into every nested tool/model trace and await all goroutines.
client.SimulateSystem(ctx, bench.SimulationOptions{...}) takes the same cases and
revisions plus CreateSession func(context.Context, any) (bench.SimulationSession, error). Use bench.SimulationInput{InitialState: ..., Turns: []any{...}} as case
input and require ExpectedState. The returned session implements Turn,
Observe and Close, each accepting context.Context. A fresh session handles
1 to 20 scripted turns; observed state is snapshotted before Close resets it.
bench.EvaluationCaseID(ctx) identifies the current case.
Timeout defaults to 30 seconds per case; set Timeout up to five minutes.
Cancel the supplied context to stop. Missing assertions/observations, unfinished
spans and timeouts are incomplete and fail Passed(). Go cannot forcibly stop an
uncooperative goroutine. Use isolated test dependencies and cooperative callbacks.
Cleanup gets a fresh five-second context, which it must honor.
Real app tests record redacted content locally, even when production tracing
is metadata-only. They do not upload reports or spend evaluation credits by
implicitly running a judge. Use synthetic test data.
Run the complete refund simulation example. It sends two refund requests, checks the number of refunds written, and demonstrates the failing implementation and its fix.

