Evaluations

Measure quality
before users do.

Build test sets, score outputs with rules, code or model graders, and gate every release on the results. Evaluations power automatic rollback.

Know before you ship.

What you get with Evaluations.

Create them by hand, import them, or capture them from real traces.

Exact match, regex, JSON schema, custom code, or a model acting as a judge.

Block a version from rolling out if it scores below the current one.

Score a sample of live traffic continuously and alert on regressions.

Do evals cost extra?+

Evals run on your chosen models and count toward normal usage.

Ship without fear.

Nothing is a black box.

Prompts deserve an IDE.

From $5 per user per month. Bring your own models. Connect everything in one click.