Skip to main content
Connecting a repository identifies systems and highlights quality risks and lower-cost model options. Open a finding for its evidence, then choose Start benching to test it. Choose View more for the explanation. A highlighted Quick scan section identifies what is not yet verified by evaluation. Cost findings put the potential model first, then show the explanation, model comparison and price sources. Use the benching action below to verify the finding.

​
What the scan can tell you

The current quick scan is a static review, not a Jev check or an executed evaluation. An unresolved input is a testing gap, not proof that your application fails.

​
Potentially lower model cost

Potentially 86% lower model cost means a candidate model has a lower price for the same input and output volume. It does not mean your production bill has fallen. For example, standard text prices checked on 19 September 2026 are: Both prices are about 86% lower for GPT-5 Mini. At an illustrative 1,000 input and 500 output tokens per call, that is about 8.75versus8.75 versus 1.25 per 1,000 calls. This example assumes identical usage, standard text requests, and no caching, batch discounts, tools, hosting or negotiated rates. Different models can use different numbers of tokens, retries or tool calls. Sources: GPT-5.2 pricing and GPT-5 Mini pricing. The expanded finding identifies both models and links to their price sources. If input and output prices fall by different percentages, Bench shows a range. Missing models and stale prices remain unknown, not zero. Quick-scan prices expire after 30 days unless refreshed.

​
Turn an opportunity into evidence

  1. Bench the original setup against system-specific criteria and cases.
  2. Test supported candidates against the same cases.
  3. Compare quality, failures and estimated model cost per call.
  4. Validate the proposed change on independent cases before rollout.
  5. Use production usage after deployment to verify the real impact.
A cheaper model is not automatically a better choice. The recommendation must meet the quality and regression checks. A quick-scan comparison also does not guarantee that the named alternative is enabled in the current candidate set. During evaluation, Bench calculates model cost from recorded input/output usage and its price catalog. Judge and test-generation spend are separate from the customer workflow’s projected cost. These are estimates, not imported invoices. See evaluation results and production checks.

​
Why there is no predicted quality percentage before testing

Repository structure and model price do not establish answer quality. A quality gain needs a baseline and a candidate evaluated on the same representative cases. A small preliminary test can support a provisional result, but it must show its sample size and cannot stand in for a full benchmark or independent validation. Jev can judge narrow questions, such as whether a reply follows a supplied policy. Its probability is not a percentage of system quality gained. In the current preview, Jev checks selected production interactions; running it automatically during repository scans or using it as the full benchmark judge is not wired up. Do not interpret a static finding as a hidden Jev evaluation.