Let’s talk about the weekend the law stopped letting the algorithm decide silently.
This Sunday — the second of August — the EU AI Act’s high-risk rules become enforceable. Not the press release version. The enforceable version.
For the first time in a major economy, an AI system that decides something important about you has to prove itself before it touches your life.
Let me ground this in a person, because the law is easier to miss than the consequence.
Picture a hiring manager who has never met the candidate the software rejected. The screen gave her a shortlist. The shortlist came from a model she has never seen, trained on data she has never audited. When someone asks why a candidate was cut, the honest answer is: the model scored them low. Nobody can say for what.
That is the world we have been living in. Scoring for jobs, credit, insurance, school admissions, rental applications — decisions about your life made by systems that keep no minutes.
From Sunday, inside the European Union, that changes for the systems on the high-risk list. Employment. Credit. Education. Access to essential services. Law enforcement. Migration. That list is the map of where the machine has been quietly deciding.
And the law does something that has never been done at this scale: it makes the machine carry the burden of proof about itself.
Before a high-risk system goes live, the provider has to show risk management, data governance, technical documentation, logging that records what the system did, and a conformity assessment that someone independent actually signed. The system has to be registered in a public EU database. And the people deploying it — the banks, the employers, the schools — have to put human oversight in place, tell you when you are being scored, and give you a right to explanation and a complaint.
That is a big deal. Read it slowly. The machine now has to keep minutes.
Here is why it matters beyond Brussels.
This is the first time the law has looked at AI the way it looks at a bridge or a drug: something that can hurt people, so something that has to be tested, documented, and traced before it is used on them. Not a feature. Not a marketing claim. A regulated instrument with a paper trail and a named party who answers for it.
And that kills the oldest excuse in the industry. “The algorithm did it.” That defence dies on Sunday.
Because the Act does not let the model be the accountable one. It names providers. It names deployers. It names the humans who put the system in place and the humans who are supposed to oversee it. When the machine is wrong — and it will be wrong — there is now a documented chain of who built it, who bought it, who ran it, and who was watching.
The invisible score finally has an owner.
Here is what people are missing.
First: this is not a tech-company problem. If you buy or use a high-risk system — and almost every bank, insurer, large employer, and lender in Europe uses one — the obligations sit on you, the deployer. The vendor can be compliant and you can still be the one holding the liability, because the law demands that you, the human institution, maintain oversight, monitor the system after it goes live, and stop using it when it drifts. Compliance is not something you purchase. It is something you run.
Second: the scramble has already begun, and most of it is theatre. Consultants are selling paperwork. Vendors are selling “AI Act ready” badges that mean nothing until an auditor actually looks. And the human oversight requirement — the heart of the whole law — is the easiest thing in the world to fake. You can put a name on a dashboard and call it oversight. You can give that person no time, no access to the model’s reasoning, and no real authority, and the box is ticked. We saw this exact trick this week with the WHO’s clinical AI framework. Oversight without authority is a rubber stamp with a pulse. The EU built the architecture. The fight now is whether the humans in the loop are real — whether they have the time, the data, and the protection to actually say no.
Third — and this is the one nobody in the industry wants to say out loud: the systems on that list have been deciding without documentation for years. The data to prove they were safe does not exist, because nobody kept it. The audit trails are empty. The risk logs were never written. So a lot of what goes live compliantly on Monday will be, in practice, a fresh start with a clean sheet — not because the systems are proven, but because the paperwork is new. Do not confuse documentation with safety. Documentation is the precondition for safety. It is not the proof of it.
So here is the operator lesson, and it is worth more than the compliance cost.
If you build or buy AI that touches people’s lives, the oversight layer is now the product. The model is the engine. The oversight layer is the law, the reputation, and the reason anyone will keep trusting you. Build the audit trail like it will be read in court, because one day it will be. Give the humans who oversee the system real authority, real access, and real protection — or the law will find the gap, and so will your customers.
And remember who this is actually for. Not the compliance departments. Not the vendors. The person who gets scored. The candidate who never gets a call. The applicant who can no longer be rejected by a machine that refuses to explain.
That is the promise of Sunday: the machine has to answer for itself.
The question is whether we keep oversight real enough that the answer means something.
Because a paper trail is only worth anything if a human is free to follow it — and free to say the machine is wrong.