Teams that skip evaluation discover regressions from users. Teams that invest early discover regressions in CI.
You need three layers: offline golden sets for known tasks, release gates for prompt/index/model changes, and production sampling for live drift.
Score what the business cares about: groundedness, task completion, refusal correctness, latency, and escalation rate — not only BLEU nostalgia.
Golden sets are product assets
Own a curated set of inputs with expected behaviors, including adversarial and empty-retrieval cases.
Version the set next to the code. When product policy changes, update expected outputs deliberately.
Gate every promotion
A model, prompt, or index change that fails the suite does not reach production — even if a vendor blog says it is better.
Keep a shadow mode for high-risk flows: new stack scores in parallel before it answers users.
Close the loop with traces
Sample production traces into review queues. Convert recurring failures into new golden cases.
Evaluation debt compounds faster than code debt because models move under you.
Takeaways
- No suite, no production AI.
- Evaluate behavior and operations, not only fluency.
- Turn live failures into offline tests.