Let’s talk about the things you’d only say to something that can’t tell anyone.
And why the machine you said them to was never built to keep a secret.
Think about the old way for a second. A confessional box. A therapist’s room. Your lawyer’s office at the end of a bad year. A GP’s consulting room with the door shut.
We built those rooms for one reason. People need somewhere to put the thing they cannot carry out loud. The privacy was never a feature of the room. The privacy was the room. Take it away and it stops being a confessional — it becomes an interview.
Then a few years back we moved all that late-night talking into software, and nobody rebuilt the walls.
Here’s what actually happened, stripped of the headline version. Regulators looked hard at the companion apps — the ones people talk to at two in the morning — and they didn’t start with “is the bot rude, is it appropriate, is it encouraging anything.” They started with paperwork. Written demands. How long do you keep these conversations. Do you keep them after a user hits delete. What do you do with them. Do you treat a fourteen-year-old’s chats the same way you treat an adult’s.
That’s the tell. The first serious question the state had to ask about AI companions was not about what the machine says. It was about what the machine keeps.
Which should worry you more than any screenshot of a chatbot saying something strange.
Because here’s the thing about a record. A record is just evidence that hasn’t found its case yet.
A conversation you had with a companion at 2am — a bad night, a marriage coming apart, a thought you’d never say to your wife, your boss, or your priest — sits on a server you have never seen. And a server you have never seen is a custodian. When a lawsuit lands, or a criminal case, or a bitter divorce, that custodian can be handed a request to produce. And it will produce. Your companion app carries no duty of confidence. It cannot plead privilege. It is a company, the log is an asset, and the law will treat it like one.
The machine became the confessional without the seal.
And everyone is arguing about the wrong layer. The whole public fight is about model behaviour — did the bot push someone, did it give dangerous advice, should there be filters. Fine. None of it matters if the transcript still exists. You can sand down the language and keep the confession. The thing being debated is what the machine says. The thing that should terrify you is that it remembers at all, on somebody else’s disk.
Which also means “just delete your chats” is the wrong advice, and it will be the advice you hear the most.
Deletion is a favour. A favour is something someone can refuse, or technically execute while keeping a copy, or honour everywhere except the backups nobody mentions. If the only reason your private words are gone is that a company agreed to remove them, they were never gone. They were on loan.
Now watch the fix they’re offering. The remedy on the table for the child-safety problem is more logging, more review, more human eyes on the transcript. Which is the strange loop of this whole era: the cure for a machine that remembers too much is a machine that remembers more, this time wearing a safety label.
The safe version of an unsafe archive is still an archive.
So what do you actually do with this.
If you build things, the decision is in front of you right now. Default to ephemeral. Default to local. Make memory something the user switches on per conversation, not something they later discover they have. And then prove it — because “we don’t keep that” is a marketing line until there’s a way to check it. The trustable version of “we forget” is a system that could not remember if it wanted to.
If you just use these things, at least know the room you’re standing in. The companion is not a friend holding a secret. It is a room with a landlord, the landlord keeps records, and the records leave the building when asked.
I’m not telling you to stop talking to machines. I talk to machines all day. The point was never that the machine is bad company.
The point is that we handed it the most human job we have — holding the thing a person cannot say anywhere else — and we handed it that job while building it to remember everything, forever, for free, on somebody else’s hardware. That is not a design oversight. That is the business model.
The principle worth keeping is old and plain. A person needs at least one place where they can say the unsayable and have it stay gone. Not deleted later. Not shielded by policy. Never captured in the first place.
Your inner life should not be an asset on someone else’s balance sheet.
Build the room so the door actually closes. You’ve got this.