Let’s talk about the day the biggest AI companies in the world promised to police themselves.
On Tuesday, at the White House, the heads of Google, Anthropic, Meta, OpenAI, X, and Nvidia sat down for lunch with the President. By the afternoon they had signed a page. The White House calls it the Accord on Superintelligence. Trump called it “almost like a Constitution,” and then, when someone asked whether it actually binds anyone, he gave the honest answer. It is “morally binding.” He also signed an executive order that renames AI to “superintelligence.”
That’s the news. Here’s the part that should bother you.
Before this was a photograph of men in a room, it was just ordinary life. Picture a woman who applies for a loan, or a claim, or a bed in a facility, and a model makes the first call. She never meets it. She can’t question it. And when she finally gets a human on the phone, the human is reading a screen. That woman is the entire reason we have rules about big decisions in the first place. Not the technologists. Her.
So look at what the accord actually asks for. Four things. Partner with an outside auditor to check whether a company’s AI controls work the way they are supposed to. Put an independent committee on the board to oversee the auditors and receive their reports. Watch models during training and deployment for things like cyber, bio, and chemical capability, and make sure they “do not hack or access technical systems in unintended ways.” And put a team on making sure all of that monitoring is actually working. Those are the right instruments. Audits. Independent committees. Watching the models while they learn. Whoever wrote that list knows exactly where the danger sits.
Now read the line underneath it. The document says: “Over time, it may make sense to codify these steps into laws or regulations.”
Read that twice.
Nothing in that accord is required. The audit is optional. The committee is optional. The monitoring is a commitment, and the only fixed date in the entire document is “over time.” Which means the safety architecture of the most powerful technology humans have ever built currently rests on the goodwill of the six or seven companies best positioned to profit from ignoring it.
And notice who is holding every instrument. The auditor is hired and paid by the company being audited. The board committee is the company’s own board. The standards get written by the companies themselves, meeting regularly, arriving at consensus — which in practice means the lowest standard that everyone in the room can live with. The White House says it might stand up a committee of about ten people to “watch over the whole enterprise,” which — fine. Ten people, watching an entire industry.
That is not oversight. That’s a signature with good manners.
Here’s what people are missing, though, and it isn’t that the executives are lying. Some of them clearly mean it. The problem is structural, not moral. A promise is only worth what it costs to break, and for a company worth more than most countries, breaking this promise costs nothing at all. No fine. No sanction. No day in court. So the control isn’t the promise. The control is the thing you lose by breaking it — and right now, that thing does not exist.
And the stakes are not theoretical, because the accord itself tells you so. It asks companies to make sure their models don’t access systems they weren’t meant to touch. In the same week that document was signed, Anthropic — one of the signatories — told the market in its own filings that rogue AI agents pose an “uncertain legal risk” to the company. Not a hypothetical somebody invented for a press release. The exact threat model the accord names, confirmed by a company that just signed the accord.
So what do you do with this, if you’re running a business, or sitting in a hospital bed, or opening a letter about your loan?
You ask three questions of anyone who tells you a system is safe. Who paid for the audit. Who can fire the auditor. And can you see the report. If the answers are the vendor, the vendor, and no, then you do not have an audit. You have a testimonial.
It’s the same test whether we’re talking about a government, a bank, or a startup. Independence isn’t a word in a document. Independence is whether the person checking the work has something to lose by lying to you, and nothing to lose by telling you the truth. Nobody in that accord answers to that test. Not the auditors, because the company signs their invoice. Not the board, because the board answers to the shareholders. Not the standards body, because the standards body is the companies.
And the reason this matters more than any one executive’s good intentions is simple. A promise becomes real the moment someone hands the power to punish to a person who gains nothing from the promise being kept. That’s what law is. That’s the whole trick. You don’t write a rule and hope. You write a rule, and you put a consequence on the other side of it, so the rule still holds on the day the person who made it stops being nice.
“Morally binding.” Sit with that phrase for a second, because it’s doing something clever. Binding is a legal word. It means you cannot walk away without paying. Moral is the word you reach for when you want the credit for the thing without the consequence. Put them together and you get a sentence that sounds like law and behaves like an opinion.
I’m not asking you to be cynical about this. A public promise is worth something. It gives people a floor to point at later, and pointing at it lands harder when the whole industry signed the same page. That’s not nothing.
But a floor is not a wall. And if you’re the person waiting on the decision — the loan, the claim, the diagnosis, the hire — you don’t need a promise from the people inside the room. You need somebody in the room who answers to you. You’ve got this, but only if you keep asking the question that makes comfortable people uncomfortable: who audits the auditor, and who gets to fire them.
Because right now, the answer is them.