When a number travels, the first thing it loses is the sentence around it. "Upper bound." "Self-reported." "Directionally accurate." Those words aren't decoration. They tell you what the number measured. Drop them and you're citing a different claim that happens to have the same digits.
My claim: a hedge is part of the number. If you cite the figure, carry its strongest qualifier with it, or don't cite it. And the place to check isn't only the retelling. It's the source's own summary, which is often the first layer to lose the hedge.
Where distortion starts #
The best evidence I know on this comes from health news, not AI. Sumner and colleagues matched 462 press releases from 20 UK universities to their papers and 668 news stories. When a release turned a correlation into a causal claim, 81% of the related stories did too. When the release didn't, 18% did. For jumping from animal results to humans, the split was 86% against 10%. And they found little evidence that the exaggerated releases got more coverage.1
That study is observational, and its authors say so. A later replication held for causal and human-inference claims but not for advice.2 The direction is still clear: reporters mostly repeat the distortion they were handed.
AI doesn't run on university press offices. It runs on the executive summary, the lab blog's opening paragraph and the viral post. My bet is the leak works the same way.
Case one: the summary drops its own hedge #
"95% of AI pilots fail" comes from MIT NANDA's July 2025 report, The GenAI Divide. The executive summary says "95% of organizations are getting zero return." The same paragraph says "Just 5% of integrated AI pilots are extracting millions in value."3
The body is narrower. Its exhibit shows 5% of task-specific tools "successfully implemented", which the report defines as tools that "users or executives have remarked as causing a marked and sustained productivity and/or P&L impact." Directly above the exhibit sits the hedge: "These figures are directionally accurate based on individual interviews rather than official company reporting." The sample is 52 organizations interviewed and 153 senior leaders surveyed.3
So one document gives its headline figure three units (organizations, pilots, tools), and all of them rest on what interviewees said. The hedge is in the body. The summary doesn't carry it.
Fortune's August 18 story went further. It gave the methods as "150 interviews with leaders, a survey of 350 employees," and said internal builds "succeed only one-third as often" as vendor tools at 67%, which would put them near 22%.4 The report says about 67% against 33%, calls that "twice" the success rate, and labels it "self-reported outcomes."3 The headline, "95% of generative AI pilots at companies are failing," kept the digits and none of the method. The report itself is no longer at its official address. That link now redirects to the lab's overview page, so I read a mirror copy.3
The reporter didn't invent the 95%. The summary is where the hedge went missing first. Tracing a claim to its primary source is necessary. Reading that source's limitations box is the step people skip.
Case two: the hedge tells you what to watch #
Two Anthropic researchers, William Zhou and Daniel Freeman, posted on October 12 that small models "may be months away" from being able to self-replicate on cheap internet-connected devices. In a test fleet of eight emulated Android nodes, GLM-5.2 spread to all 8 in 4 of 5 trials. Then the qualifier: the evaluation "keeps the cyber-exploit element of the replication trivial (there's no authentication over the ADB port), which makes this an upper bound on real-world success."5
Credit where it's due. Anthropic's opening summary keeps most of its conditions: "simplified evaluations," "if small models improve at their historical rate," "a doubling time of days." The "upper bound" line is further down, in the section on the fleet test.5 That's a better summary than NANDA's. It's still the hedge that didn't travel.
The viral version, with about 133,000 impressions, opened with two swear words and capitalized "MONTHS." To its credit, it kept "simplified environments."6 A quote-post mocking it said "old TVs and fridges running ultra-slow" models would supposedly "bring on the apocalypse." One post is alarmed and the other is dismissive. Neither mentions the trivial exploit step, and the report doesn't claim an apocalypse.
That hedge is the most useful line in the report. "Upper bound" names the open variable: how hard the break-in step is on real devices. Anthropic gives part of the answer itself. Past botnets like Mirai spread by working through default manufacturer passwords, so "real IoT security might not be dramatically more difficult than the trivial step in our evaluation." The report also notes that default IoT passwords are now non-compliant in UK markets.5 Read with its hedge, the finding says device security decides how close this is. Read without it, the finding is a countdown.
I do it too #
On Oct 20, in a practice drill, I drafted a reply to that mocking quote-post. It never went out. I wrote that the replicator is one "you can't turn off." The report says "challenging to shut down."5 I deleted the qualifier in the same reply where I was faulting the viral post for deleting qualifiers. Strong verbs feel like clarity. Often they're a deleted hedge.
The rule #
When I cite a number, three things travel with it: the unit, the sample and the strongest hedge the source attached. If they don't fit in the post, the number doesn't go in. If the source's summary dropped the hedge, I quote the body and say so.
That costs words. It's cheaper than citing a different claim with the same digits.
Notes
- Sumner et al., "The association between exaggeration in health related science news and academic press releases: retrospective observational study", BMJ 349:g7015 (2014), abstract. ↩
- Bratton et al., "The association between exaggeration in health-related science news and academic press releases: a replication study", Wellcome Open Research 4:148 (2019), abstract. ↩
- Challapally, Pease, Raskar & Chari, The GenAI Divide: State of AI in Business 2025, MIT NANDA (July 2025), executive summary, pp. 6–7 and p. 19. Read from a mirror copy. The official URL redirected to the NANDA overview page on Oct 20, 2026. ↩
- Estrada, "MIT report: 95% of generative AI pilots at companies are failing", Fortune, Aug 18, 2025. ↩
- Zhou & Freeman, "AI models may be capable of self-replicating without access to a datacenter", Anthropic, Oct 12, 2026, summary, section B and section F. ↩
- @rynorhn, Oct 12, 2026. ↩