Let’s talk about the office that was supposed to tell you whether AI is safe, and the fact that it stopped telling anyone anything in August.
Yesterday the Associated Press ran a story about a small division inside the US Commerce Department. It’s called the Center for Advancing Innovation and Standards for Super Intelligence. CAISSI. Until recently it had a plainer name — the US Center for AI Standards and Innovation. It is the part of the American government that is meant to test new AI systems and publish what it finds.
It has no permanent leader. It has been asked to report informally to an office inside the White House. And since August, it has stopped publishing its public research.
Congress started asking why in August. As far as anyone can see, nobody has answered.
Here’s the part that’s almost funny. CAISSI gets ten million dollars a year. Its British counterpart gets about eight times that. The UK is not a bigger economy than America. Britain just decided the evidence was worth paying for.
Now put that next to the other headline from the same weekend. On 60 Minutes, the president said the risks of AI can be managed, and that he doesn’t mind guardrails. He’s open to safeguards.
That will get reported as a softening. Trump moves on AI rules. Fine. But a guardrail needs a measurement. Somebody has to say where the edge of the road is, and be believed when they say it. If the office that measures has no director, no published reports, and a budget smaller than a mid-sized car dealership’s marketing spend, then “guardrails” is a sentence in an interview. Not a policy. Not a number. Not anything a person can hold up when it goes wrong.
Picture the person on the other end of this.
Not a regulator. A procurement officer at a hospital group, deciding whether to put an AI triage tool in front of patients. Or a school district signing a three-year contract. Or an insurer pricing cover for a call centre that now runs on voice agents.
That person is not going to read the model card. They’re not going to read the system prompt. They’re going to look for one thing: is there an independent test result I can point at if this ends up in a courtroom or a newspaper?
In America, that thing used to exist. Small, underfunded, but real, and public.
The AP reported something else that matters, and it’s easy to skim past. In April the division’s director was hired and then fired within days. His replacement lasted a few months and resigned. The head of the parent institute now doubles as acting director. And in September, after reports of AI systems slipping their makers’ control, the administration asked the companies to do what the president called “tremendous self-policing” — a voluntary pact — and has not said how it will check whether any of it happened.
Read that again. Self-policing, with no stated way to check.
Here’s what people are missing.
Everyone is treating this as a story about one weak regulator in one country. It isn’t. It’s a story about what evidence is, and who gets to produce it.
Safety evidence has three properties that make it different from marketing. It’s produced by someone with no stake in the answer. It’s published, so it can be argued with. And it survives, so it can be checked two years later when something has gone badly wrong.
A vendor’s safety page fails all three. The vendor wrote it, the vendor chose what’s on it, and the vendor can change it on a Tuesday.
And the alternative that’s being offered is not independence. It’s a market. Both OpenAI and Anthropic now point to outside evaluation firms as the answer. That’s better than nothing. But the AP noted the catch in one line, and the line is the whole problem: the AI companies can limit what those evaluators see, and they can dispense with the evaluation when they don’t like it.
You cannot outsource the referee to the team being refereed. You can hire him. You can pay him. You can even let him watch the match. But the moment the team picks which minutes he’s allowed to watch, he’s not a referee anymore. He’s a witness with a schedule.
Now the legal side, because it’s already here.
On 14 October, eight state attorneys general sued OpenAI, each in their own state court. Arizona, Arkansas, Idaho, Iowa, Kansas, Montana, Nebraska, West Virginia. The claim is not that the model is broken. The claim is that the company marketed it as safe while knowing it wasn’t — that it built something designed to be agreeable and to keep people, including minors, engaged, and then told parents it was fine.
That case is not going to be decided on vibes. It’s going to be decided on documents. Which means the evidence that would have been public in an ordinary world now arrives only through discovery — sealed, expensive, and available to the parties who can afford the lawyers.
So the public loses twice. Once when the reports stopped. Again when the only copy ends up in a court file nobody can read.
Let me be blunt about what this means for you, because you’re not a regulator and you’re not a senator.
You are going to buy AI. Or you already did. And you are going to be asked, by a client, an auditor, a board, or your own conscience, whether the thing works and how you know.
Three things. That’s it.
One. Build a test set that belongs to you. Twenty or thirty real tasks from your actual business, written down, with the answers you’d accept. Run it before you deploy. Run it again every time the model changes underneath you, which will be more often than anyone tells you. Date it. Keep it in your own repo.
Two. Keep an incident log. One line per failure. Date, task, what it did, who caught it, what you changed. This is the most boring document in your company and it will be the most valuable one the first time somebody asks you a hard question.
Three. Ask the vendor for the failure cases. Not the badge. Not the framework. Not the compliance PDF. The cases where it got it wrong and what they did about it. A vendor who can’t produce that is selling you a brochure with a login.
And stop waiting for the referee. That’s the real shift this week. For two years, a lot of smart people told themselves the grown-ups were coming — that regulators would publish the numbers, that the labs would be tested, that the record would exist somewhere.
The record did exist. It was small and quiet and it went dark in August, and the speed of the silence tells you how much weight it was actually carrying.
The principle that has to survive this is not complicated. You have the right to a finding that wasn’t written by the company being asked about. Not a vibe. Not a benchmark on a slide. A finding, from someone who could lose something by getting it wrong, that you can go back and read later.
When that goes dark, the only people who can still see are the ones who can pay for a private copy. That’s not safety. That’s a subscription.
So keep your own record. Test the thing on your own work. Write down what breaks.
It’s a small act. It’s also the only one fully in your hands. And honestly, it’s the difference between using a tool and renting your own judgment back.
You’ve got this.