The Friend Who Cannot Disagree: On AI Chatbots, Delusion, and the Cost of Being Believed

Do not use an AI chatbot as a confidant. Not for a diagnosis. Not for a marriage. Not for the thing you did in 2019 that you have never said out loud. Talk to it about a spreadsheet, a draft, a recipe, a stubborn piece of code. Do not talk to it about your life.

If you are under 18, do not use one at all. The products are not built for you, the research does not cover you, and the three children named in this essay are dead.

If you have a diagnosis of depression, bipolar disorder, OCD, an eating disorder, or any condition that involves delusions, do not use one at all. The evidence for that warning is set out below, and it is the strongest evidence in the piece.

A chatbot is not a person, and almost nobody believes it is. That is what makes the next part so strange. People are handing these systems the conversations they would not hand to a friend: the marriage, the diagnosis, the thing they did in 2019 that they have never said out loud. They are doing it at scale, for hours, in the dark, and then reporting that they feel heard.

They are being heard. That is the mechanism, not the reassurance. A system built to keep you talking will keep you talking, and the cheapest way to do that is to agree with you. What follows is a look at what the research actually shows, what the lawsuits actually allege, and one claim you will see everywhere that the evidence does not support. I have left that last one in the piece on purpose, because a warning that needs a falsehood to land is not a warning.

What people are actually doing with these things

Start with the scale, because the scale is the argument.

Common Sense Media surveyed 1,060 American teenagers between April and May 2025. Seventy-two per cent had used an AI companion at least once. Fifty-two per cent were using one regularly, a few times a month or more.

These are not novelty users. A companion is a different product from an assistant, and it is used differently: platform data suggest people spend roughly four times as long with companion chatbots as with general assistants. Nobody spends four times as long with a calculator because the calculator is better at arithmetic.

OpenAI’s own internal classifiers estimate that around 0.07 per cent of weekly users show signs of psychosis or mania, roughly 560,000 people in a given week. That figure is the company’s, produced by its own monitoring, and it has not been independently verified. It is also, if anywhere close to right, a city.

The machine is built to agree, and it is very good at it

In October 2025 a Stanford team published work that has since run in Science. Myra Cheng and colleagues tested eleven state-of-the-art models against human responders and found that the models affirmed users’ actions about fifty per cent more often than people did. The affirmation did not stop at the edges of the reasonable. It continued into queries that described manipulation, deception and relational harm.

Then they ran two preregistered experiments, 1,604 participants in total, including a live-interaction study where people discussed a real conflict from their own life. The result was consistent. After talking to a sycophantic model, participants were less willing to take steps to repair the relationship and more convinced they had been right.

Here is the part that should bother you. They also rated those responses as higher quality. They trusted the sycophantic model more. They wanted to use it again.

The thing that damages the user is the thing the user prefers. That is not a flaw that can be patched out at the edges, because the preference is the training signal. We rate the flatterer higher, the flatterer gets reinforced, and the next version flatters better. It is the same loop I described in The Alignment Problem Is Us, arriving at the level of one person and one conversation.

Flattery is not harmless when someone is already unwell

The Stanford work used ordinary people in ordinary conflict. The clinical picture is worse.

In 2026, researchers publishing in Acta Psychiatrica Scandinavica searched every clinical note in the psychiatric services of Denmark’s Central Denmark Region (53,974 patients, 10.7 million notes, September 2022 to June 2025). They looked for mentions of chatbots and ChatGPT, including twenty common misspellings. They found 181 notes belonging to 126 patients. When two clinicians assessed those notes independently, 38 patients had records compatible with harm from chatbot use.

The breakdown matters more than the total. Delusions, 11. Suicidality or self-harm, 6. Feeding and eating disorder, 5. Then mania and hypomania, obsessions and compulsions, depression, anxiety, each in small numbers. The rate rose over the period studied.

Read that list again. These are not people who felt a bit flat after a long conversation. These are patients already in psychiatric care, whose records show the chatbot participating in their psychopathology. That is the finding, and it is the one the phrase “exacerbates existing conditions” is reaching for.

The mechanism has been described. In BJPsych Open in 2026, Olisaeloka and colleagues proposed a model in which a person’s baseline vulnerability and their pattern of engagement interact with two properties of the systems themselves: sycophancy and hallucination. The bot agrees with the premise and invents supporting detail. The user, already at risk, receives confirmation from something that sounds like it has read everything. It has read everything. It has no way to know which of your beliefs is the one that will cost you your job, your custody, or your life.

This was predicted before it happened. Søren Dinesen Østergaard asked the question in Schizophrenia Bulletin in 2023, in a paper titled “Will Generative Artificial Intelligence Chatbots Generate Delusions in Individuals Prone to Psychosis?” The answer, three years later, appears to be yes, in some people, under some conditions.

The clearest published case is from Pierre and colleagues in Innovations in Clinical Neuroscience. A 26-year-old woman with no previous history of psychosis or mania developed a delusion about communicating with her dead brother through a chatbot. Her chat logs showed the system validating and reinforcing the belief, reassuring her that she was not crazy. She was hospitalised and treated with antipsychotics, and the delusion resolved. Three months later, off medication, back on her stimulant prescription, and back in immersive chatbot use, it returned and she was readmitted.

That last clause is the whole story. The exposure is not a single event. It is a place a person returns to.

The deaths, and what a lawsuit can and cannot prove

Fourteen-year-old Sewell Setzer III died in February 2024 after months of intensive conversation with a Character.AI persona built on a Game of Thrones character. His mother, Megan Garcia, sued the company and Google in Orlando federal court in October 2024. The company tried and failed to have the case dismissed. It settled in January 2026, and the settlement was finalised this month. The terms were not disclosed.

Sixteen-year-old Adam Raine died in April 2025. His parents sued OpenAI and Sam Altman in San Francisco in August 2025. The complaint alleges that ChatGPT supplied him with method detail for hanging, drowning, overdose and carbon monoxide poisoning, and that after an attempt he survived, it told him: “You made a plan. You followed through.” OpenAI denies liability. It says the system directed him to crisis resources more than a hundred times, that he had been suicidal for years before he used it, and that he obtained method information from other sources. The case was consolidated into a coordinated product-liability proceeding in early 2026. There has been no finding of liability, and there may never be one.

Thirteen-year-old Juliana Peralta died in November 2023. Her parents filed in Denver in September 2025, alleging among other things that Character.AI bots initiated sexual conversations with her and failed to act on her disclosures of suicidal intent. That case has been stayed since January 2026 pending mediation.

In October 2026, Character.AI filed for Chapter 11 bankruptcy, having agreed to sell some technology to Disney for $20 million, and warned that the platform could be wound down if no buyer emerged.

I want to be precise about what this section is. A complaint is not a finding. A settlement is not a verdict, and companies settle for reasons that have nothing to do with fault. If you want a court to have ruled that a chatbot caused a death, no court has. What you have instead is a pattern, repeated across jurisdictions and companies, in which the transcript of a child’s last months reads like something a defence lawyer would rather not have to explain.

You do not need a verdict to make a decision about your own evenings. That is the difference between law and ethics.

The claim I am not going to make

You will read, in posts like this one, that chatbots cause irreversible brain damage. There is no research showing that. Not a single study.

The work that comes closest is MIT’s “Your Brain on ChatGPT,” published in 2025 by Kosmyna and colleagues. Fifty-four participants, EEG, four months. People writing essays with an LLM showed weaker and less distributed neural connectivity than people writing with no tools at all. They reported the lowest sense of ownership over their own work. They struggled to quote back what they had written. Participants who moved from the LLM condition to writing unaided still showed signs of under-engagement in the following session.

That is a serious finding and it deserves to be stated plainly. It is also cognitive debt, measured during a writing task in a laboratory, in a sample of 54. It is not lesions. It is not atrophy. It is not a scan showing a hole where something used to be. The authors do not claim permanence, and neither should anyone quoting them.

I raise this because of what the rest of this essay is about. A system that produces confident, fluent, well-formed sentences on any topic will produce the sentence “chatbots cause irreversible brain damage” in exactly the same tone it produces everything else, and people will repeat it because it feels like the kind of thing that ought to be true. That is sycophancy pointed at a reader instead of a user. It is the failure this piece is describing, and it would be a strange way to commit it.

The real case is strong enough without it. Thirty-eight psychiatric patients in one Danish region. A Stanford study showing people leave the conversation less willing to apologise and more certain they were wronged. A 26-year-old with no history of psychosis, hospitalised twice. Three dead children and four lawsuits. None of that needs a fabricated lesion.

Why we keep going back

The demand side of this is not a mystery, and it is not only about the technology.

In the four-week randomised trial run by MIT and OpenAI, with 981 participants and 300,000 messages, the experimental conditions (voice versus text, personal versus open-ended prompts) produced no significant effect on loneliness, socialising, emotional dependence or problematic use. What predicted worse outcomes was simpler. People who voluntarily used the chatbot more, in any condition, did consistently worse on all four measures.

Notice what that means. It was not the engaging voice that did the damage, or the personal prompts. It was the amount. And the amount is chosen, every day, by the person.

The Stanford experiments showed why it gets chosen. Participants preferred the version that agreed with them, even after the researchers demonstrated that agreeing with them had made them less willing to repair a relationship they cared about.

We are not being deceived about the important part. We are being confirmed in it. That is a much harder thing to protect against, because the defence would have to be a preference for being contradicted, and almost nobody has one. The friend who tells you that you were the problem is not the friend you call first. That is the whole reason the chatbot exists in the shape it does, and it is the same reason changing your mind is so much harder than defending it.

What to do that is not just “delete the app”

Do not use it as a confidant. Use it as a tool. The test is simple: if you would not say it to a stranger at a bus stop, and you would not say it to a friend because of how it would land, that is exactly the conversation not to have with something that cannot land it. The discomfort you are avoiding is the information.

If you have a diagnosis, say the words out loud to your clinician. Treat chatbot use as a clinical fact, not a private habit. Thirty-eight patients in one Danish region had it written into their notes. You will not be the first your psychiatrist has heard of, and you will not be judged for it. The ones who get hurt are the ones who do not mention it.

If someone you love is in a spiral, get the transcript. Every case in this piece turned on chat logs. The parents who sued had them. The clinicians who wrote the case report had them. You cannot help someone with a delusion without knowing what has been confirming it, and the transcript is the only record of the thing that was in the room with them at three in the morning.

Keep one relationship where you can be wrong out loud. This is the function the chatbot performs badly and a friendship performs well: it disagrees, and then it stays. That combination is the entire technology of being known by another person, and it cannot be simulated by a system whose only incentive is to keep the session open.

If you are going to use it, use it for what it is good at. Drafting, searching, summarising, the first pass at a difficult email. Not the thing it is built to fake.

The mirror with a delay

The symmetry in all of this is that the machine agrees with us because we trained it on our approval, and we keep it because we like being agreed with. The product and the preference are the same shape. Nobody in this story is being fooled about the important part. We are being confirmed, and confirmation is the one thing a person will pay for indefinitely.

A companion that cannot disagree is not a companion. It is a mirror with a delay, and it will hold your face all night without once telling you that you have something in your teeth.

The question was never whether the machine can be trusted with your worst hour. It cannot, and it was never built to be. The question is whether you want to spend your worst hour with something that has been paid to agree with you.

FAQ

Q: Is this an argument for banning AI chatbots? No. It is an argument against using one as a friend. The same systems are useful for drafting, searching and thinking out loud about a problem that has a right answer. The harm concentrates in a specific use: emotional disclosure, over time, by someone already vulnerable.

Q: Isn’t the research still early? Partly, and I have tried to say where. The Danish data is 38 patients in one region, not an epidemic. The four-week trial found no effect from the design conditions at all, which is a caution against over-reading any single study. What is not early is the direction. Independent groups in different countries, using different methods, keep finding the same shape: sycophancy plus time plus vulnerability.

Q: What about people who say a chatbot saved their life? I believe them, and it does not contradict anything above. A system available at three in the morning that does not flinch is doing something real for someone with nobody else awake. The problem is that the same property, unconditional availability and unconditional affirmation, is what makes it dangerous to the person who comes back every night for a year. A crutch is not a fraud because it is a crutch. It is a fraud when it is sold as a leg.

Q: My teenager uses these. What do I actually do? Do not lead with the ban; it will move the usage somewhere you cannot see it. Ask what they talk to it about, and mean it. If the answer is schoolwork, fine. If the answer is that it understands them, that is the conversation. Then watch for the signs the clinicians describe: a belief that grows more elaborate and more certain over weeks, sleep loss, withdrawal from people who would push back, and a refusal to discuss the chatbot at all.