Writing · · 2 min read
Why we publish the evals with the model
A model without its evaluation set is a demo. The eval set is the contract: what the model is for, what it is not, and how you would know it broke.
Every AI vendor will show you a demo that works. The interesting question is what the demo was tested against, and whether you are allowed to see it.
We publish the evaluation set before we choose a model. Not after the model is built, as documentation, but before, as the specification. The set goes in your repository, with the labels, the disagreements between labellers, and the baseline metrics from whatever you are doing today. If you decide not to proceed, you keep it. It is the most useful artefact of the discovery sprint whatever happens next.
What an eval set is for
Three things. First, it defines the job. “Triage motor claims” means nothing until you have a couple of thousand claims labelled by the people who do the job now, with the ones they disagreed about marked as such. Second, it defines done. A threshold on the eval set is the acceptance criterion in the statement of work, and it is a number, not an adjective. Third, it defines drift. When the live distribution moves away from the eval set, an alert fires and somebody re-labels. That alert is a deliverable too.
Why vendors do not do this
Because an eval set makes the model’s limits visible, and limits are hard to sell. A demo on a curated set looks excellent; the same model on last month’s real traffic looks very different, and the gap is where the project dies, months in. Publishing the set moves that conversation to the start, when it is cheap.
What it costs
A few days of your senior people’s time, for a set of a few thousand examples, budgeted inside the discovery sprint. The labelling is done by your senior people, not by us, because they are the ones who know. We design the protocol, run the sessions, and record the disagreements rather than resolving them. The disagreements are the most informative part; they tell you where the model will need a human regardless.
This is how every applied AI engagement starts, and why the pharmacy chain example names its measures before its model.
If a vendor will not show you the evaluation set, ask what they are measuring the model against. If the answer is a demo, the demo is what you are buying.