Let’s talk about the person who ends up holding the risk after everyone else has written it out of their contract.
Picture the Tuesday afternoon a founder signs off on an AI system for her business. Forty people, decent revenue, no legal department worth the name. She doesn’t read the whole thing. Nobody does. But somewhere in the six documents in front of her there’s a shape.
The model provider’s terms cap liability at the fees paid in the last twelve months. For a business her size that’s a few thousand rand against a mistake that could cost millions. The platform she buys through disclaims all warranties, including fitness for a particular purpose. The integrator’s contract makes her the party responsible for validating outputs. The insurance renewal has a new endorsement on it that takes generative AI injury out of cover. Her own customer contract says she’s liable for errors caused by her subcontractors and her software. And her staff are indemnified by the company for acts in the course of employment.
Six documents. Six parties. Every single one of them did the normal, reasonable, lawyer-approved thing, which was to move the risk one name down the line.
The line ends at a person in a chair, holding a pen.
That’s the thing I want to talk about. Not AI safety. The plumbing of consequence.
There’s a term for this in insurance and structured finance: the residual. The residual claimant is whoever eats the losses after everyone with a prior claim has been paid. It’s a dull technical phrase until you notice that in a lot of AI deployments the residual is a single human being, and nobody sat her down and told her.
Risk transfer isn’t new, and it isn’t evil. Every contract in commerce does some version of it. What AI changes is attribution. When the machine produces a wrong number, a wrong clinical note, a wrong denial, a wrong price, the error is genuinely diffuse. It came out of a model trained on data nobody fully audited, running on infrastructure nobody in the room controls, prompted by someone who has since left the company, evaluated against a benchmark that was measuring something else.
Every party upstream can say “we didn’t do that” and be telling the truth as far as their own logs go. That’s what makes it structural rather than cynical. Diffuse causation doesn’t spread responsibility around. It concentrates it on the only party who can’t plausibly point elsewhere — usually whoever was closest to the person who got hurt, or whoever has a licence to lose.
So you get a result that has nothing to do with how good the model is. The party with the least negotiating power carries the most exposure. The founder, not the lab. The clinic, not the vendor. The contractor who used the tool, not the company that sold it.
And here’s the part that gets skipped. That arrangement doesn’t just feel unfair. It makes the system worse.
A person who carries the consequence but not the authority behaves in a completely predictable way. She slows everything down. She adds a review step that reviews nothing. She gets a second signature from someone who also can’t decide, so there are two names on the decision and neither one can defend it. Then she quietly routes around the whole thing and uses the old spreadsheet, because the old spreadsheet is the one object in the building she can explain in a room.
That’s what liability without authority produces. Not caution. Theatre. And theatre is expensive, and it catches almost nothing — it just distributes the blame before the error lands.
You’ve seen the signature of this in ordinary engineering teams for years. The person who will be blamed for the outage is never the person allowed to change the architecture. So nothing changes until it breaks.
The principle I’d put on the wall is this: consequence, authority, information and cover have to travel together. If you hold the risk, you have to hold the decision. If you hold the decision, you have to be able to see what you’re deciding on. And if you’re going to be blamed for overriding the machine, somebody has to stand behind you when you do it.
A lot of the regulation being written right now requires a human in the loop and then stops. It mandates the signature. It says very little about whether the person signing has the time, the access, or the protection to say no. A required signature with no authority behind it isn’t oversight. It’s a place to park the liability.
So before you deploy anything that touches money, health, or somebody’s file, do the boring work. Draw the risk map. For each party — model provider, platform, integrator, insurer, you, your client — write down who eats the loss when it goes wrong. If the answer is “us” on every row, you’ve learned something about the deal you just made.
Then ask what you’re actually allowed to do about it. If the honest answer is “nothing, we just have to trust it,” you are not buying a tool. You’re absorbing somebody else’s risk for free, and the price should reflect that. Most people never price it. That’s the negotiation nobody runs, and it’s usually worth more than the discount they spent three weeks chasing.
Get the override in writing while you’re at it. Not a promise on a call. A clause, a named person on the other side, and a path to reach them during an incident rather than during business hours.
And if you’re the individual inside the company rather than the owner — the clinician, the analyst, the engineer who signs — the question gets narrower and harsher. Do I actually have the authority this signature implies? If not, put it in writing before the incident, not after. One email that says “I’m being asked to approve outputs I can’t independently verify and I don’t have the access to do so” is worth more than six months of good intentions.
Here’s why this belongs here and not in a contracts newsletter.
We keep talking about AI as if the danger is the machine getting too powerful. The nearer danger is that the machine makes consequence unownable. Every layer gets a document that says it wasn’t them, and we end up with systems that shape people’s lives where nobody is answerable for the shape.
A person who can’t be held to anything isn’t free. They’re just unaccountable. And a world where nobody is accountable is a world where the only thing left deciding is power.
The residual is where the human still is. Which means the residual is also the one place a human can still stand.
So before the next signature, find out who’s holding it. If it’s you, make them hand over the authority that goes with it. And if they won’t — that tells you exactly what they think the risk is worth.