· 4 min read · northitech
Why we publish the evals with the model
A model without its evaluation set is a demo. The eval set is the contract: it says what the model is for, what it is not for, and how we would know if it stopped working.
Every AI vendor will show you a demo that works. The interesting question is what the demo was tested against, and whether you are allowed to see it.
We publish the evaluation set before we choose a model. Not after the model is built, as documentation, but before, as the specification. The set goes in your repository, with the labels, the disagreements between labellers, and the baseline metrics from whatever you are doing today. If you decide not to proceed, you keep it. It is the most useful artefact of the first two weeks whatever happens next.
What an eval set is for
Three things. First, it defines the job. “Triage motor claims” means nothing until you have a couple of thousand claims labelled by the people who do the job now, with the ones they disagreed about marked as such. Second, it defines done. A threshold on the eval set is the acceptance criterion in the statement of work, and it is a number, not an adjective. Third, it defines drift. When the live distribution moves away from the eval set, an alert fires and somebody re-labels. That alert is a deliverable too.
Why vendors do not do this
Because an eval set makes the model’s limits visible, and limits are hard to sell. A demo on a curated set looks excellent; the same model on last month’s real traffic looks very different, and the gap is where the project dies in month four. Publishing the set moves that conversation to week two, when it is cheap.
What it costs
A few days of your senior people’s time, for a set of a few thousand examples, budgeted inside the first four weeks. The labelling is done by your senior people, not by us, because they are the ones who know. We design the protocol, run the sessions, and record the disagreements rather than resolving them. The disagreements are the most informative part; they tell you where the model will need a human regardless.
If a vendor will not show you the evaluation set, ask what they are measuring the model against. If the answer is a demo, the demo is what you are buying.