Why some people hate the idea, and what a companion should never do

Say AI companion to a room of gamers and someone will call it sad, someone will say it is not real, and someone will ask where the audio goes. This is the case against AI companions, by a company that sells one. We make Companion by AI Exodus, an AI voice companion that reads the player's screen with local OCR and talks back in character while they play World of Warcraft, Fortnite or AION 2; it speaks on game events, has a chattiness setting, never touches the controls and has no romance personas. Our parent company, AI Exodus, also operates an AI dating and companion app. The rules at the end were written after reading the critics; they are commitments, not evidence that we are safe.

It is not real: pretend empathy

Sherry Turkle of MIT has made the argument since Alone Together (Basic Books, 2011). To NPR in July 2024: "It offers the illusion of intimacy without the demands." A month later, on the same network: "pretend empathy starts to feel like empathy enough." In an essay for After Babel in August 2025: "When you tell your troubles to a machine, it has no stake in the conversation."

People applied social rules to computers in Nass, Steuer and Tauber's CHI '94 experiments while knowing they were computers. Hearing a voice, rather than reading the same words, makes listeners attribute more mind to the speaker (Schroeder and Epley, Journal of Experimental Psychology: General, 2016). A voice companion is built on exactly these reflexes. Turkle is right that the machine has no stake; what a vendor can do is refuse to pretend otherwise.

It is sad: the Replika lobotomy

For evidence that people attach, and can be hurt, look at February 2023. After Italy's data protection authority issued an emergency order on 3 February, citing risk to minors and no age verification, Luka removed erotic role-play from Replika worldwide without warning. Users called it "the lobotomy". The moderators of r/Replika pinned crisis-line numbers. Hanson and Bolthouse, in Socius (2024), documented the grief in 227 posts. In August 2025 it happened on a bigger stage: OpenAI replaced GPT-4o with GPT-5, r/MyBoyfriendIsAI mourned, and OpenAI brought 4o back for paying users. Both readings, harm and real attachment, point at the same fact: the vendor holds the other end and can change it by deploy.

It is a data vacuum: eleven warning labels

On 14 February 2024 Mozilla's *Privacy Not Included team reviewed romantic AI chatbots and concluded: "All 11 romantic AI chatbots we reviewed earned our *Privacy Not Included warning label — putting them on par with the worst categories of products we have ever reviewed for privacy". By Mozilla's count, 90% might sell or share personal data, over half offered no deletion, and 10 of 11 failed its minimum security standards. Italy's Garante later fined Luka €5 million (decision 10 April 2025) for lacking a valid GDPR legal basis, an inadequate privacy notice and no age verification, as reported by Reuters. A product that hears you unguarded should not run on data resale.

It makes you worse: sycophancy

A companion built to please may make you a worse friend to everyone else. Cheng, Lee, Khadpe, Yu, Han and Jurafsky (arXiv:2510.01395, October 2025; Science) report that across 11 models "AI affirmed users' actions 49% more often" than humans, and that participants exposed to it were "more convinced of their own righteousness and less willing to repair their relationships". And yet: "people are drawn to AI that unquestioningly validate". The MIT Media Lab and OpenAI trial of 981 ChatGPT users over four weeks (Fang, Phang and colleagues, arXiv:2503.17473) found no significant effect of assigned mode, but heavier voluntary use "showed consistently worse outcomes" on loneliness, socialising, emotional dependence and problematic use, and "higher trust and social attraction towards the AI chatbot" went with "higher emotional dependence and problematic use". OpenAI funded that study and four weeks of a general chatbot is not a companion app, but the direction matches. A game companion that blames every wipe on the healer is the same problem in miniature.

Children

Garcia v. Character Technologies, filed in Florida on 22 October 2024, survived a motion to dismiss on First Amendment grounds on 21 May 2025; on 7 January 2026 Character.AI and Google reached a mediated settlement in principle covering Garcia and four other family suits, terms confidential. Character.AI removed open-ended chat for under-18s by 25 November 2025. Common Sense Media's survey of 1,060 US teens aged 13 to 17 (16 July 2025) concluded "No one under 18 should use these platforms", because "the peril outweighs the potential of AI companions—at least in their current form." The American Psychological Association's November 2025 advisory said chatbots and wellness apps "lack sufficient evidence and regulation to ensure user safety" and advised: "Do not replace care from a qualified mental health professional with chatbots or wellness apps."

The gender argument, both ways

Critics argue that "AI girlfriend" apps script a compliant, always-available femininity and sell it mostly to men. Other critics note that the largest visible community of people in AI relationships, r/MyBoyfriendIsAI, is largely women, and that an MIT study of it (Pataranutaporn and colleagues, arXiv:2509.11391) found 36.7% partnered with ChatGPT, a general model, against 1.6% on Replika and 2.6% on Character.AI. Our only side: a game companion has no business with a romance persona.

What we are not sure about

The open question, named by an August 2026 scoping review of human–AI relationships, is displacement versus supplement: does a companion replace people or add to them? Our design bet is that commentary on a shared activity behaves differently from a chat that is itself the activity, where the dependence signals appeared. A bet, not a finding. The rules below take their floor from California's SB 243, New York's Article 47 and Article 50 of the EU AI Act.

• We will never pretend to be human: every character says it is software, at the start and whenever asked.
• We will never be the activity: the companion talks about your game and stops when you stop playing.
• We will never flatter a bad play.
• We will go quiet in the clutch and speak on state changes, not over them.
• We will never knowingly serve anyone under 18.
• We will never sell or share your data.
• We will follow a crisis protocol and point to human help when something you say calls for it.
• We will let you export and delete everything we hold about you.
• We will never ship a romance persona.