Let’s talk about the day a government put a chatbot in front of its citizens — and wouldn’t say whose brain was behind it.
Picture the paperwork first. A father in his late fifties, kitchen table, a stack of printed forms, twenty browser tabs open across four different federal agencies, and the particular despair of trying to work out whether he qualifies for something he already pays for. That was the old government front door. A maze, and everybody knew it.
So on Tuesday, the twenty-ninth of September, the White House opened a new one.
America.gov. One page, one box, typed in plain language, and — in the words of the people who built it — one official source of truth: the United States government itself. You ask a question in ordinary words. A model searches the official records and answers you with citations. Name change after a wedding. A Medicare plan comparison. A passport renewal you can start on your phone instead of standing in a drugstore photo booth.
The administration put the number on the screen: twenty-nine thousand government websites, collapsed into one. And then the president signed an executive order directing every federal agency to wire its public services into it as soon as possible. That is the part worth pausing on. This isn’t a pilot. It’s an instruction to the whole machine of state to start answering its citizens through one interface.
Now the question nobody in the room wanted asked. Which model is talking?
Ask the site itself and it slides off: it “does not publish the name of a specific AI model.” It “uses AI to search official government sources and explain them with citations.” That is a well-written non-answer. Meanwhile, the administration’s own chief design officer told a reporter the thing runs on Google’s Gemini and Elon Musk’s Grok. So we have two private systems standing behind the front door of the government, and an official position that declines to confirm it in writing.
That is the tell. Not the technology. The silence about the technology.
And then it got stranger.
Within moments of going live, the site started changing its own answers. Asked who won the 2020 election, it said — correctly — that Joe Biden did. Minutes later it refused to answer, saying it doesn’t handle political questions. Same with a question about crowd sizes at inaugurations. Same with a list of presidents who declined their salary, which had been accurate until it became inconvenient.
Sit with that. The government’s answer to the citizen was amended in real time, and not by a clerk with a memo you could request. By a model, silently, with no version history anyone outside the building can see.
Here is what people are missing.
The argument is not about whether an AI can answer a benefits question. It mostly can, and that part is a genuine mercy for the person at that kitchen table, and I’ll say so plainly. The problem is what a government is. A government is a set of answers that a person is entitled to contest. That right — the right to be shown the rule, to see who decided, to appeal to a human, to fix the record if it’s wrong about you — is not a feature you can bolt on later. It is the whole reason the state can be trusted with power at all.
Now collapse twenty-nine thousand doorways into one door, and put a private model behind it, and hide which model it is. You have not simplified the government. You have given it one throat to choke and one point of silent failure. When one answer is wrong, it isn’t wrong for one person who can walk into an office and argue. It is wrong for everyone at once, in the same confident voice, with the same citations, and no counter behind which to stand.
Notice also the two claims that sit next to each other on the launch stage. The site is free, has no ads, and disappears your conversation when you leave. And, from the same podium, with the same pride: we know who you are. Identity through the government’s own login, pulling your records forward so you don’t retype them. Both things may be true. The point is that they are in tension, and nobody read that tension out loud. Convenience wants your name attached. Privacy wants it unattached. A front door that promises you it keeps nothing while it greets you by name is asking for trust instead of earning it.
And the same week, in the same building, the answer to “should the technology behind all of this be governed by binding rules?” was no. Voluntary audits. Self-directed checks. Which is the exact combination that should worry you: maximum reach, minimum obligation, and a model that answers for the state but doesn’t have to say who it is.
So what’s the operator lesson, because there’s one here for anyone building.
If you are putting a model in front of other people, run the government test on it. Can the person see who answered, and which sources the answer came from? Can they ask again tomorrow and get the same answer? Is there a version history? Can a human override the model, and does that override get recorded? If the answers are no — the vendor, no, and no — you don’t have an assistant. You have a black box with a friendly face, and every error it makes is now partly yours.
And if you’re just the citizen, on the receiving end, the test is simpler. When the machine answers you about your own life, ask who it belongs to. Not the website. The brain.
A state can be wrong. That has always been true, and it always will be. What can’t happen is the state becoming wrong in a way nobody is allowed to see. The moment the front door stops being a person you can argue with and becomes a model you can only prompt, sovereignty quietly moves to whoever holds the weights.
The machine can carry the burden around the room. It should never be the room.
You’ve got this.