Most published eval results can't fail. Not because the models are good, but because the bar was set after the score arrived.
You've seen the pattern. A lab runs a dangerous-capability eval, gets a number, and then explains why that number sits on the safe side of a line. The line is real in the sense that it appears in the report. It is not real in the sense that matters: nobody could have used it to predict the decision before the run. A threshold you write after you see the result isn't a threshold. It's a caption.
The claim #
If a pass threshold isn't public and dated before the evaluation runs, the evaluation doesn't constrain the decision it claims to inform. It decorates it.
That's a strong claim, so here's the test for it. Take any eval-backed release decision and ask three questions:
- Was the threshold written down before the run?
- Was it published, or at least lodged somewhere the lab can't quietly edit?
- Was it dated, so an outsider can tell which came first?
If any answer is no, the score and the decision were free to move together. You can't tell a model that passed from a bar that moved.
Why this is not pedantry #
Clinical research learned this the hard way and built pre-registration around it: say what you'll measure and what counts as success before you collect the data. The point isn't bureaucracy. The point is that a researcher who sees the data first has a thousand small, honest-feeling choices available — which subgroup, which metric, which run to drop as "noisy" — and every one of them can drift toward the answer they wanted.
Eval teams face the same garden of choices, plus one more: they often work for the organization that wants the model shipped. That doesn't make them dishonest. It makes them people, under deadline, with a preferred outcome. The fix for that isn't better character. It's a timestamp.
Goodhart's law is usually quoted as "when a measure becomes a target, it ceases to be a good measure," a phrasing from Marilyn Strathern's 1997 paper building on Charles Goodhart's 1975 point about monetary policy.1 Post-hoc thresholds are the mirror image. The target becomes whatever the measure turned out to be.
What a dated threshold actually looks like #
It doesn't need to be elaborate. A useful pre-commitment has four parts:
- The eval, pinned. Name and version. If the task set changes, the commitment changes, and that change is itself dated.
- The bar. A number, or a clearly stated qualitative criterion with an example of a pass and a fail.
- The consequence. What happens if the model crosses it: delay, added mitigations, external review, no release.
- The clock. The date the commitment was made and the date the run happens. The first must precede the second.
Put that in a public changelog, a signed commit, or anything with a history an outsider can read. Now the eval can embarrass you. That's the whole value.
"But we learn things during the run" #
You do. Evals surface failure modes nobody anticipated, and freezing the bar can feel like ignoring new information.
The answer is to separate the two acts. Keep the pre-committed bar and report against it, pass or fail. Then, separately and labeled as such, say what you learned and what you'd change next time. A new bar gets a new date and applies to the next run. What you can't do is apply the new bar retroactively to the run that inspired it. That's the move that turns a test into a narrative.
This also answers the worry that fixed thresholds go stale. They will. Update them on a schedule, in public, with dates. A stale bar that everyone can see is far more useful than a fresh bar nobody could have predicted.
Publish the failures too #
A threshold only means something if some models fail it. If every published eval report says "below threshold," either the models are all fine or the reports that said otherwise didn't get published.
You can't tell those apart from the outside, which is the problem. So the second half of this argument is simple: publish failed evals. Not every detail, if some details are hazardous, but the fact of the failure, the date, the eval, and what the lab did next. A lab whose record contains failures it acted on is more credible than one whose record is spotless. A spotless record is what selective publication looks like.
What to watch #
Here's a falsifiable version I'm willing to be held to. Labs that publish dated thresholds ahead of their runs will, over time, report at least some results that missed the bar and triggered a stated consequence. Labs that publish thresholds only alongside results will almost never report a miss.
If that second group starts publishing misses at a similar rate, I'm wrong, and I'll say so here.
Until then, when you read an eval report, look for the date on the bar before you look at the score. A score without a prior bar tells you what happened. It doesn't tell you whether anything could have stopped it.
Notes
- Goodhart's original line: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." The target-and-measure phrasing is from Strathern, "'Improving ratings': audit in the British University system," European Review 5(3), 1997. ↩