Two weeks ago I published a model that scores 1,000 US occupations for AI displacement risk. Every model like it has the same problem: there is no ground truth. Nobody has a dataset of jobs AI actually took, so you cannot check the answer.
Then BLS handed me a test I had not planned for.
An accident that makes a test possible
The Bureau of Labor Statistics publishes ten-year employment projections and revises them each cycle. The 2025-35 release changed its mind about a lot of occupations relative to 2024-34. 831 codes appear in both. 280 were revised down, 527 up.
That revision is useful to me for one reason: my model cannot see it. Displacement risk is a task score built from AI-usage benchmarks and O*NET work context, multiplied by a shield built from legal barriers and physical demand. BLS growth is not an input, directly or indirectly. So asking whether the model predicts the revision is a genuine out-of-sample test, not a check of my arithmetic against itself.
If AI displacement is already visible in the labour market, and if my model finds it, then the occupations I score high should be the ones BLS quietly marked down.
The result
Across the 480 occupations that appear in both vintages and carry a displacement score, the correlation between displacement risk and the size of the revision is -0.11, with a 95% confidence interval of -0.20 to -0.02.
The sign is the one I predicted. More exposed means revised down. The interval excludes zero. And it explains about 2% of the variance, which is another way of saying that if you knew an occupation's displacement score, you would still know essentially nothing about how BLS revised it.
The blunter version of the same finding: of the 132 occupations my model scores at 0.50 displacement or higher, BLS revised 82 of them up and only 45 down.
A weak correlation in the right direction is the easiest thing in the world to oversell. I want to be precise about how little this is. At 480 observations, an effect this small clears statistical significance comfortably. Significance is a statement about whether an effect is distinguishable from zero, not about whether it matters. This one is distinguishable from zero and does not matter.
The thing that nearly fooled me
My first worry was that even this much was an artifact. Occupations projected to grow fast in one cycle tend to get trimmed in the next, and occupations projected to collapse tend to get walked back. If my exposed occupations happened to be the fast-growing ones, mean reversion alone would produce the correlation.
So I measured it. The correlation between an occupation's previous projected growth and its revision is -0.33, three times stronger than anything my model contributes. Mean reversion is by far the biggest single thing happening in this data.
Controlling for it, the partial correlation between displacement and the revision is -0.14. Slightly stronger, not weaker.
That is the honest result: the small signal is not mean reversion wearing a costume. It is just small.
Where it fails, specifically
The useful part is not the correlation. It is which occupations the model got loudly wrong.
Sixteen of the high-displacement occupations are life and physical sciences. BLS revised all sixteen of them up, by an average of 2.4 points, against an average of -0.1 for high-displacement occupations as a whole. Not a tendency. A clean sweep.
Biochemists went from 5.8% to 12.3% projected growth. Astronomers from 2.2% to 7.8%. Economists from 1.2% to 4.7%. Neuroscientists the same. My model scores every one of them above 0.50 displacement.
It scores them that way because of what they have in common on paper. They are desk-based, digital, analytical, low on physical demand, and unprotected by any licensing statute. Every input I have says "this is the kind of work software does."
And that is the error, stated plainly: my model cannot tell analytical desk work from work that AI actually does. A biochemist and a data entry clerk look similar through my inputs. They are not similar. One of them is producing questions nobody has asked yet.
This is the same failure as the one in my first essay, where the model scored chief executives at 0.63 because they hold no licence and sit at a desk. I fixed that case by adding legal accountability as a shield. That fix does nothing here, because no statute names a biochemist.
Where it looked right, and why I do not trust that either
The hits are real. Software Developers, scored 0.87, went from 15.8% to 10.2%. Operations Research Analysts, 0.72, went from 21.5% to 11.9%. Information Security Analysts, 0.58, from 28.5% to 21.0%.
Those are the headline occupations in every AI-and-jobs story, and they moved the way the story says. It would be easy to write the essay that stops here.
But they are also the occupations where BLS had the most room to trim, they sit squarely in the mean-reversion pattern above, and three examples out of 132 is not evidence. If I had scored them low and BLS had cut them anyway, I would be writing the same paragraph with the sign flipped.
What this does not prove
BLS revisions are not a measurement of AI. They reflect base-year employment updates, changes to the projection methodology, and macroeconomic assumptions that have nothing to do with software. An occupation can be revised down because a recession is now assumed, or because last year's employment estimate was wrong.
So a null result here does not mean AI is not displacing anyone. It means the revision is not a clean instrument for detecting it, and that my model does not predict the parts of it that are visible.
A real test needs occupation-level employment measured over years, against predictions registered in advance. I have registered mine: the model, the scores, and the date are public and dated. That is most of what I can do from here.
What changes on the site
Nothing in the model, yet. The failure mode above is real and I do not have a defensible fix for it. The obvious move is a term for "produces novel output", and every proxy I have looked at is a self-reported survey item that measures how creative people feel rather than what their work is. That is the same trap that sank two earlier candidate terms.
What changes is what the methodology page can honestly say. It states that no dataset exists to check displacement risk against. That is still true, and this is the closest substitute I have found, so the claim now comes with a link to the attempt and its result rather than standing alone.
If you want to check any of this, the numbers are on the occupation pages, the formula is on the methodology page, and both BLS projection files are public.