LLM evals are how you turn "this feels better" into "this scored 8 points higher on our test set." They are the unit tests of AI systems: a fixed dataset of inputs, an expected-behavior spec, and a scorer that tells you whether a change helped or hurt.
Why it matters in production
LLM output is non-deterministic, so you cannot assert exact string equality the way normal tests do. Without evals, every prompt tweak is a gamble and every model upgrade is terrifying — you have no way to know what you broke. Evals convert that fear into a number, and a number you can gate a deploy on. This is exactly why we run evals as CI: a change that regresses the score never merges.
The common mistake
Waiting until "later" to build evals, then never building them. The second mistake is scoring only the final answer while ignoring retrieval — a RAG system can produce a fluent wrong answer because the retrieval step failed, and an answer-only eval never catches it. Measure each stage.
How we actually use it
We build a small, honest eval set early — real inputs, real failure cases — and grow it every time production surprises us. Where an answer is open-ended, we score it with an LLM as a judge rather than brittle string matching. Evals are also the safety net that makes a model migration a measured decision instead of a leap of faith.