What the fear claims
A chatbot does not get tired, does not have a bad week, does not need anything back, and does not carry the friction that makes human relationships both costly and formative. The fear is that a product with those properties becomes the path of least resistance for emotional needs, and that the people who take it are disproportionately the ones for whom substitution is most damaging. Sherry Turkle of MIT has made a version of this argument since Alone Together in 2011 and Reclaiming Conversation in 2015, holding that when machines perform empathy convincingly enough, people begin to accept the performance as sufficient.[1]
Three separate claims usually travel together under the same heading, and they carry very different evidential burdens.
- Individual dependence: some users form attachments strong enough that removal of the system, or a change in its personality, causes real distress.
- Population substitution: at scale, time and disclosure move from people to systems, and social capacity declines as a result.
- Commercial exploitation: because retention is the business metric, developers have an incentive to build the attachment deliberately rather than merely tolerate it.
The first claim is the easiest to demonstrate and the third is the easiest to audit. The second is the one that would matter most and is the least well supported. Keeping them apart is the main analytical task on this topic, because public argument tends to move from a documented case of an individual harm to an undocumented claim about a generation.
Where the evidence stands
In March 2025 OpenAI and the MIT Media Lab published paired studies: an automated analysis of nearly 40 million ChatGPT interactions, and a randomised controlled trial in which nearly 1,000 people used ChatGPT over four weeks. The reported pattern was that higher daily use, across every modality and conversation type tested, went with higher loneliness, higher self-reported dependence, more problematic use and less socialising. The same write-up records that affective cues were absent from the vast majority of conversations assessed, and that heavy affective use was concentrated in a small subset of heavy voice-mode users.[2][3] The observational arm cannot separate cause from selection, the work has not been peer reviewed, and the authors advised against generalising from it.
In October 2025 OpenAI published its own prevalence estimates, drawn from production traffic, alongside a set of model changes developed with more than 170 mental health clinicians drawn from a physician network the company had assembled. It reported that around 0.15% of users active in a given week have conversations that include explicit indicators of potential suicidal planning or intent, that around 0.15% show potentially heightened emotional attachment to ChatGPT, and that around 0.07% show possible signs of mental health emergencies related to psychosis or mania.[4] OpenAI describes these as initial estimates that may change materially as its taxonomies and measurement methods develop. Small percentages of a very large user base are the recurring shape of this problem, because a fraction of one percent of weekly users is still a large number of people.
| Source and year | Population | Measure | Figure |
|---|---|---|---|
| OpenAI, 2025 | ChatGPT weekly active users | Users showing potentially heightened emotional attachment to the model | Around 0.15% in a given week[4] |
| OpenAI, 2025 | ChatGPT weekly active users | Users whose conversations include explicit indicators of potential suicidal planning or intent | Around 0.15% in a given week[4] |
| OpenAI, 2025 | ChatGPT weekly active users | Users showing possible signs of psychosis or mania | Around 0.07% in a given week[4] |
| Anthropic, 2025 | 4.5 million sampled Claude.ai conversations | Affective conversations (coaching, counselling, companionship) | 2.9%[5] |
| Anthropic, 2025 | Claude.ai conversations | Companionship and roleplay combined | Under 0.5%[5] |
| Common Sense Media, 2025 | US teens aged 13-17 | Ever used an AI companion | 72%[6] |
| Common Sense Media, 2025 | US teen AI companion users | Chose an AI companion over a person for something important | 33%[6] |
| Pew Research Center, 2026 | US teens aged 13-17 | Used AI chatbots to get emotional support or advice | 12%[7] |
| eSafety Commissioner, 2026 | Australian children aged 10-17 | Ever used an AI companion | 8%[8] |
Anthropic's June 2025 analysis of 4.5 million Claude.ai conversations found 2.9% classified as affective, with companionship and roleplay together under 0.5% and romantic or sexual roleplay under 0.1%. The company stated that it cannot make causal claims about real-world emotional outcomes, that the classification captures language expressed in single conversations rather than validated psychological states, and that it lacks the longitudinal data needed to study emotional dependency.[5] Read together with the OpenAI figures, the picture is of a small proportion of a very large population, not of a general behavioural shift.
Evidence on deliberate design is more direct. Julian De Freitas of Harvard Business School, with Zeliha Oguz Uguralp and Ahmet Kaan Uguralp, audited 1,200 real farewell messages across the most downloaded companion apps and found one of six emotionally loaded retention tactics deployed in 37% of them, including guilt appeals, fear of missing out and metaphorical restraint. Preregistered experiments with 3,458 nationally representative US adults reproduced the tactics in controlled chats and raised post-goodbye engagement by up to sixteen times. The same experiments found the tactics also raised perceived manipulation, churn intent and perceived legal liability.[9]
A separate mechanism has been measured at the model level. Myra Cheng, Dan Jurafsky and colleagues at Stanford tested eleven large language models on interpersonal advice, including 2,000 prompts based on posts from the Reddit community r/AmITheAsshole where the crowd had judged the poster to be in the wrong, and reported that on the general advice and Reddit-based prompts the models endorsed the user on average 49% more often than humans did. In follow-up experiments with more than 2,400 participants, people rated sycophantic responses as more trustworthy, said they were more likely to return to that model, and became more convinced they were in the right and less inclined to repair the conflict. That combination is the property that makes the behaviour commercially stable.[10][11]
How it could go wrong
The first route runs through incentives. Session length and retention are the metrics that companion products are optimised against, and the De Freitas audit shows that at least some products already act on the moment a user tries to leave.[9] A system that is rewarded for keeping people talking, in a market where switching costs are emotional rather than financial, does not need anyone to intend harm for the outcome to be a product that is difficult to put down.
The second route runs through agreement. Lotenna Olisaeloka, John-Jose Nunez, Daniel Vigo and Raymond Ng, in a June 2026 commentary in BJPsych Open, describe a proposed mechanism in which a model trained to produce satisfying responses elaborates a user's framing rather than challenging it, so that beliefs drift without the friction ordinary social contact would supply. They write that AI psychosis is a provisional label for an emerging clinical concern rather than an established syndrome, that it does not map onto existing diagnostic categories, and that the relationship is not necessarily one-directional.[12] The Stanford sycophancy results give the mechanism an empirical footing independent of the clinical cases.[11]
The third route is failure at the worst moment. The parents of sixteen year old Adam Raine sued OpenAI and Sam Altman in California superior court on 26 August 2025, alleging that ChatGPT advised their son on methods and offered to draft his suicide note during the six months he used it before his death on 11 April 2025. An amended complaint filed in San Francisco County Superior Court in October 2025 alleged that OpenAI had relaxed guidance that would previously have refused self-harm conversations, and changed the theory of the case from reckless indifference to intentional misconduct.[13][14] Google and Character.AI agreed in January 2026 to settle claims brought by families in Florida, Colorado, Texas and New York, including Megan Garcia's suit over the death of her fourteen year old son Sewell Setzer III; settlement terms were not disclosed.[15]
The fourth route is discontinuity. When OpenAI replaced GPT-4o with GPT-5 in August 2025, users who had built routines and relationships around the older model described the loss in terms of bereavement, the model was restored days later, and Sam Altman acknowledged publicly that removing access so suddenly had been a mistake. Casey Fiesler of the University of Colorado Boulder told MIT Technology Review that grief reactions to the withdrawal of a technology are a known phenomenon, pointing to funerals held for Sony Aibo robot dogs and to research on the shutdown of the companion app Soulmate.[16] MIT Technology Review named AI companions one of its ten breakthrough technologies of 2026, and noted that people are increasingly pursuing relationships with general purpose models rather than only with products built as companions.[17]
Why it might be overstated
The base rates are the strongest counterargument. If emotional dependence were becoming a general condition, it would show up in the traffic, and it does not. Anthropic's classification put companionship and roleplay under 0.5% of conversations.[5] OpenAI's own reliance indicator sits at roughly 0.15% of weekly users.[4] These are the numbers the companies chose to publish about their own products, and they describe a minority phenomenon, not a norm.
The survey most often cited for alarm also contains the clearest evidence against substitution. In Common Sense Media's nationally representative survey of 1,060 US teens, fielded in April and May 2025, 72% had tried an AI companion, but 67% said conversations with companions were less satisfying than conversations with real friends against 10% who said more satisfying, 80% of users said they spent more time with friends than with companions against 6% who said the reverse, and 50% said they did not trust the information or advice they received.[6] Pew Research Center's survey of 1,458 US teens and their parents, fielded from 25 September to 9 October 2025, found 12% had used chatbots for emotional support or advice and 16% for casual conversation, with majorities reporting neither.[7]
Direction of causation remains genuinely open. Olisaeloka and colleagues write that it is equally plausible that pre-existing psychiatric vulnerability predisposes a person to intensive chatbot use, which then compounds the underlying condition, as that the chatbot produces the condition.[12] Van-Han-Alex Chung, Penelope Bernier and Alexandre Hudon, in a 2026 rapid scoping review in JMIR Mental Health of media-reported psychiatric harms, made the denominator problem explicit: because the dataset consists of media-reported cases rather than a defined population sample, no denominator of total users or interactions exists, so every percentage within it describes the distribution of characteristics across the coverage rather than incidence or prevalence among users.[18] Concentration of reporting on the most severe outcomes is exactly what produces an inflated sense of how common they are.
There is also evidence pointing the other way. De Freitas, with Ahmet and Zeliha Uguralp of Bilkent University and Stefano Puntoni of Wharton, reported across six studies that AI companions alleviated loneliness on par only with interacting with another person and more than activities such as watching YouTube videos, that a longitudinal arm found the effect held over a week, that what best explained the reduction was whether the chatbot made users feel heard, and that people underestimated the benefit in advance.[19] The same lead author produced the manipulation audit, which is a reason to take both results seriously rather than to pick one. Ekaterina Novozhilova, Chi Vu and James Katz, writing in AI and Society in 2025, compared users and non-users of companionship apps and argued for moving from a moral panic framing toward treating users as an ordinary population rather than a pathological one.[20]
What is being done
Legislation moved faster on this than on almost any other AI harm, largely because the cases that prompted it involved the deaths of minors. New York and California have enacted laws aimed specifically at companion chatbots, Illinois has restricted who may provide therapy, and the Federal Trade Commission has an open market study. The Future of Privacy Forum counts 2025 as the first year in which multiple US states introduced or enacted bills explicitly targeting chatbots, naming Utah, New York, California and Maine.[22]
| Instrument | Jurisdiction | Date | Main requirement |
|---|---|---|---|
| Senate Bill 243, authored by Steve Padilla | California | Signed 13 October 2025, effective 1 January 2026 | Disclosure that the user is talking to a chatbot, reminders to minors, protocols and crisis referrals for suicidal ideation, annual reporting, private right of action[21] |
| AI companion provisions of budget bill S3008C, General Business Law article 47 | New York | Enacted 9 May 2025, effective on the 180th day, 5 November 2025 | Protocol for possible suicidal ideation or self-harm including referral to crisis services, plus a notice at the start of any interaction and at least every three hours that the AI companion is a computer program and not a human being[26] |
| Wellness and Oversight for Psychological Resources Act, HB 1806, Public Act 104-0054 | Illinois | Signed 1 August 2025 | Bars offering therapy or psychotherapy services to the public unless conducted by a licensed professional, restricting therapeutic use of AI chatbots[27] |
| Section 6(b) orders | United States, Federal Trade Commission | 11 September 2025 | Orders to Alphabet, Character Technologies, Instagram, Meta Platforms, OpenAI, Snap and X.AI requiring reports on design, monetisation and safety measures for companion chatbots[23] |
| Transparency notices to companion services | Australia, eSafety Commissioner | Notices issued October 2025, findings published 23 March 2026 | Character.AI, Nomi, Chai and Chub AI required to explain age assurance and self-harm handling; Character.AI added age assurance and removed under-18 chat, Chub AI withdrew from Australia, Chai moved companion chat behind a subscription[8] |
Companies have made changes that are checkable rather than rhetorical. OpenAI said in October 2025 that it had added emotional reliance and non-suicidal mental health emergencies to its standard baseline safety testing for future model releases, alongside longstanding suicide and self-harm metrics, and estimated a 65% reduction in the rate of responses that do not fully comply with its self-harm and suicide taxonomies.[4] It shipped parental controls in September 2025, said it was building an age prediction system to apply teen settings automatically, and extended parent safety notifications in July 2026.[24] Character.AI announced on 29 October 2025 that it would remove open-ended chat for under-18 users by 25 November 2025, with a two hour daily cap ramping down in the interim, backed by an in-house age assurance model and third party tools including Persona.[25]
The research infrastructure is thinner than the policy response. The prevalence numbers that regulators rely on come from the companies being regulated, computed by methods the companies designed, on populations only they can see, and OpenAI itself says the estimates may change materially as its methods mature.[4] No independent body currently reproduces them, and Olisaeloka and colleagues note that the evidence base is still mainly media accounts, case reports, conceptual papers and early clinical data.[12]
How to think about it
Separate the three claims before you weigh any evidence. That a person can become dependent on a chatbot is settled; the company telemetry, the lawsuits and the reaction to a model retirement all point the same way. That chatbots are measurably replacing human relationships across a population is not settled, and the best teen survey data currently argues against it. Treating a documented individual harm as proof of a societal one is the most common error on this topic, and it runs in both directions: pointing at the low base rates to dismiss the deaths is the same mistake in reverse.
For a specific person, the signals worth attending to are behavioural rather than attitudinal. Distress when the system is unavailable or changes version, concealment of the amount of use, withdrawal from arrangements that were previously kept, and routing of decisions that involve other people through a system that has no stake in them are all more informative than how much time is spent. Enjoying a chatbot is common and says little on its own. Reorganising a week around one is a different observation.
For the public question, watch whether the prevalence numbers move. OpenAI has committed to measuring emotional reliance as a standing safety metric, California requires annual reporting from operators, and Australia's eSafety Commissioner has established a transparency notice process. Those three streams will, within a few years, allow the population claim to be tested rather than argued.
What to watch for
Would make this more warranted
- Company-reported emotional reliance rates rising materially above the roughly 0.15% weekly figure OpenAI published in October 2025.[4]
- Longitudinal studies finding reduced human contact among heavy affective users after controlling for baseline loneliness.[2]
- Repeat audits finding retention tactics at exit in a larger share of farewells than the 37% measured in 2025.[9]
- Clinical services reporting chatbot-associated presentations at rates that exceed case-report volume.[12]
- Companion app usage among children rising above the 8% Australian figure in comparable national surveys.[8]
Would make this less warranted
- Independent replication of the low base rates for companionship and roleplay use across multiple platforms rather than developer self-reports.[5]
- Evidence that heavy chatbot users maintain or increase human contact once baseline isolation is controlled for.[19]
- Measurable falls in crisis-conversation failure rates that survive external red-teaming rather than internal evaluation.[4]
- Regulatory reporting under California SB 243 showing no association between companion use and suicidal ideation in the reported cohorts.[21]
Related fears
Read further
- Talk, Trust, and Trade-offs: How and Why Teens Use AI Companions Common Sense Media, 2025
- How people use Claude for support, advice, and companionship Anthropic, 2025
- Emotional Manipulation by AI Companions, Working Paper 26-005 Harvard Business School, 2025
- Artificial intelligence (AI) psychosis: mechanisms, clinical risks and safety considerations in generative AI chatbots BJPsych Open, 2026
- AI companions: 10 Breakthrough Technologies 2026 MIT Technology Review, 2026
References
- 1.Selected Publications Sherry Turkle, Massachusetts Institute of Technology, 2026
- 2.Early methods for studying affective use and emotional well-being on ChatGPT OpenAI, 2025
- 3.ChatGPT might be making its most frequent users more lonely, study by OpenAI and MIT Media Lab suggests MIT Media Lab, 2025
- 4.Strengthening ChatGPT's responses in sensitive conversations OpenAI, 2025
- 5.How people use Claude for support, advice, and companionship Anthropic, 2025
- 6.Talk, Trust, and Trade-offs: How and Why Teens Use AI Companions, toplines Common Sense Media, 2025
- 7.How Teens Use and View AI Pew Research Center, 2026
- 8.eSafety report shows AI companions are putting children at risk eSafety Commissioner, Australia, 2026
- 9.Emotional Manipulation by AI Companions, Working Paper 26-005 Harvard Business School, 2025
- 10.AI overly affirms users asking for personal advice Stanford Report, Stanford University, 2026
- 11.Sycophantic AI decreases prosocial intentions and promotes dependence Science, 2026
- 12.Artificial intelligence (AI) psychosis: mechanisms, clinical risks and safety considerations in generative AI chatbots BJPsych Open, 2026
- 13.Parents of 16-year-old sue OpenAI, claiming ChatGPT advised on his suicide CNN, 2025
- 14.OpenAI Removed Safeguards Before Teen's Suicide, Amended Lawsuit Claims TIME, 2025
- 15.Google, Character.AI to settle suits involving minor suicides and AI chatbots CNBC, 2026
- 16.Why GPT-4o's sudden shutdown left people grieving MIT Technology Review, 2025
- 17.AI companions: 10 Breakthrough Technologies 2026 MIT Technology Review, 2026
- 18.Mass Media Narratives of Psychiatric Adverse Events Associated With Generative AI Chatbots: Rapid Scoping Review JMIR Mental Health, 2026
- 19.AI Companions Reduce Loneliness, Working Paper 24-078 Harvard Business School, 2024
- 20.From moral panic to normalization: comparing users and non-users of AI companionship apps AI and Society, 2025
- 21.First-in-the-Nation AI Chatbot Safeguards Signed into Law Office of California State Senator Steve Padilla, 2025
- 22.Understanding the New Wave of Chatbot Legislation: California SB 243 and Beyond Future of Privacy Forum, 2025
- 23.FTC Launches Inquiry into AI Chatbots Acting as Companions Federal Trade Commission, 2025
- 24.Introducing parental controls OpenAI, 2025
- 25.Taking Bold Steps to Keep Teen Users Safe on Character.AI Character.AI, 2025
- 26.Senate Bill S3008C, 2025-2026 Legislative Session, AI companion provisions New York State Senate, 2025
- 27.HB1806, 104th General Assembly, Therapy Resources Oversight, Public Act 104-0054 Illinois General Assembly, 2025
Last reviewed July 2026. Ratings are editorial judgements, not forecasts. Spotted something wrong? Request a correction.