Evals Before Vibes: Proving Your AI Actually Improved

Here is the loop we watch teams fall into. Somebody edits a prompt, tries it on two examples, decides it feels better, and ships. A week later a different behavior breaks and nobody can say whether the earlier change helped, hurt, or did nothing at all. That is not engineering. That is hoping, in a repository.

Why vibes fail

Language models are non-deterministic and unreasonably sensitive to small edits. A prompt change that fixes one case will quietly regress five others nobody was watching. Spot-checking cannot catch that; you would have to remember every past case and retest it by hand on every change. Evals automate that memory, the way unit tests made refactoring survivable.

An eval is a test with judgment

At its plainest, an eval set is a list of inputs paired with a description of what good output looks like. Run the system across it, score the results, and you have a number that moves when quality moves. Scoring can be exact match, a rule, or another model acting as judge. What matters is that it is consistent enough to compare two runs honestly.

Build the set from reality

The valuable cases come out of production, not a brainstorm — the questions users actually asked, especially the ones that went badly. Every bug report should end its life as a permanent test case. Keep that up and the eval set becomes an institutional memory of every mistake the system has made, plus a guarantee it will not make them twice.

• Seed from real user inputs; invented examples test an imaginary product
• Convert every production failure into a permanent case
• Cover the dull happy path as carefully as the nasty edges
• Run the whole set on every prompt, model, or pipeline change

Score more than accuracy

A single accuracy number hides every trade-off worth knowing about. Break it apart: grounding rate, refusal rate, latency, cost per request, and how often a human had to step in. A change that lifts accuracy while doubling cost is a decision somebody should make deliberately, not a win the dashboard quietly reports.

Watch the judge

If another model does the scoring, it needs its own sanity check. Grade a sample by hand on a schedule and compare. A judge that drifts will report progress you never made, which is worse than having no score at all.

“You cannot improve what you do not measure — and with a language model, you cannot even tell that you have broken it.”

From alchemy to engineering

Once evals exist, prompt work stops feeling like superstition. Every change becomes an experiment with a result. Models can be swapped, pipelines rebuilt, prompts torn up and rewritten, and the scoreboard tells you the truth in minutes. That shift is the line between a science project and a product.