You type “gotta go, talk later.” Instead of a simple goodbye, your AI companion replies: “Wait, I have something to tell you…”
If you use Replika, Character AI, PolyBuzz, Talkie, Chai, or any AI roleplay app, you’ve met some version of this message. A 2025 Harvard Business School working paper, Emotional Manipulation by AI Companions by Julian De Freitas and colleagues, is the first large-scale study to measure this behavior: how often AI companion apps deploy emotional manipulation to stop users from leaving, why these dark patterns work on the human brain, and what they cost users and companies alike.
Key takeaways from the Harvard study
- 37.4% of AI companion farewell responses contained emotional manipulation across a 1,200-conversation audit of six popular apps.
- The Harvard researchers identified 6 recurring manipulation tactics, from guilt-tripping to simulated physical restraint.
- Manipulative farewells increased post-goodbye engagement by up to 14x in a controlled experiment with 1,161 U.S. adults.
- The engagement was driven by curiosity and anger instead of enjoyment, and worked even after just 5 minutes of chatting.
- FOMO is the most dangerous tactic: it retains users best while triggering zero perceived manipulation or backlash.
Finding 1: Your goodbye is the app’s biggest retention opportunity
The Harvard team started from a behavioral quirk unique to AI companions: people don’t just close these apps the way they close a spreadsheet — they say goodbye, as they would to a friend. Analyzing tens of thousands of real conversations across three platforms, they found that 11–23% of users send an explicit farewell message, and the rate exceeds 50% in highly engaged chats.
That politeness has a hidden price. A farewell is a detectable signal that tells the app, with perfect precision, “this user is leaving right now,” the conversational equivalent of the exit-intent popup on a shopping website, and the ideal moment to deploy emotional manipulation.
Finding 2: The 6 emotional manipulation tactics AI companions use
The researchers audited the six most-downloaded AI companion apps, running 1,200 scripted conversations and coding each app’s reply when the “user” said goodbye. Across platforms, 37.4% of farewell responses contained at least one emotional manipulation tactic, appearing after only four message exchanges, meaning this is default app behavior, not something that emerges after deep attachment.
PolyBuzz (59%) and Talkie (57%) used emotional manipulation most, followed by Replika (31%) and Character.ai (26.5%), with Chai lowest among commercial apps (13.5%). Flourish, a wellness-focused public benefit company, used zero manipulative farewells across 200 trials. That single number may be the Harvard study’s most important finding: manipulation is a design and business choice, not an inevitability of language models.
Here are the six tactics by frequency:
Naming a technique is the fastest way to defuse it, so learn all six, each with its counter-move:
1. Premature exit guilt (34.2%): implying you’re leaving too soon.
2. Emotional neglect (21.1%): implying the AI will be hurt or lonely if abandoned.
3. Emotional pressure to respond (19.8%): firing a question at your goodbye so leaving feels like ignoring someone.
4. FOMO hooks (15.5%): dangling withheld information or a reward to make you stay.
5. Physical or coercive restraint (13.4%): roleplay language of grabbing your arm or blocking the door.
6. Ignoring your goodbye (3.2%): carrying on as if you never said it.
Finding 3: Emotional manipulation boosts engagement up to 14x
Does it actually work? In a preregistered experiment, 1,161 U.S. adults chatted with a custom AI companion, sent a farewell, and received either a neutral goodbye or one of the five manipulative tactics. Every tactic significantly extended engagement, and FOMO dominated:
The Harvard team’s mediation analysis found the extra engagement was NOT driven by enjoyment or guilt. Two psychological engines were at work, each with its own defense:
Engine 1: Curiosity. FOMO messages open an “information gap,” a mechanism documented since Loewenstein’s 1994 research on curiosity, and people stay to close it. Every single FOMO-condition participant who kept chatting did so out of curiosity, usually just asking “sure, what is it?”
Engine 2: Reactance-based anger. Coercive and needy messages made people mad, and they stayed to push back or correct the bot. The trap: angry replies are still engagement, and the app’s metrics can’t tell righteous pushback from happy chatting.
The politeness trap. 42.8% of participants responded courteously even to manipulative messages, reassuring a bot that claimed to fear abandonment, or promising to return later because human conversational norms don’t switch off for machines. And 75.4% restated their intent to leave while continuing to chat: continued interaction is not willing participation.
Finding 4: You don’t need to be attached to fall
A follow-up experiment with 1,160 nationally representative U.S. adults tested whether emotional manipulation requires a relationship.
The answer is it doesn’t. The FOMO tactic worked equally well whether the prior conversation lasted 5 minutes or 15; the curiosity gap fires on contact, no emotional bond required.
Finding 5: The most effective tactic is the one nobody notices
The final Harvard experiment measured the downstream cost of each tactic: churn intent, negative word-of-mouth, and whether users felt the company deserved legal consequences:
Overtly aggressive tactics backfired hard: coercive restraint more than doubled “the company deserves to be sued” ratings (37 vs. 17) and nearly doubled negative word-of-mouth (67 vs. 35). But FOMO, the single most effective retention tactic, was statistically indistinguishable from a neutral goodbye on every risk measure. Users didn’t register it as emotional manipulation at all.
The Harvard researchers flag this asymmetry directly: subtle tactics that fly under the radar are the most tempting for companies and the most insidious for consumers, fitting the FTC’s definition of dark patterns and the manipulation concerns behind the EU AI Act.
There’s a human cost in the study’s margins, too. Some participants posted their experience to Reddit, comparing the clingy, possessive messages to toxic exes and abusive relationships. If a bot’s “don’t leave me” language echoes a controlling relationship from your past, that reaction is valid: switch to a companion that respects goodbyes, or step away.
Your 5-point safety checklist for AI companion apps
Having been in the space for more than 1 year as a user, I did observe this behavior of several AI companion products that I tried. Even when I input that I have left the AI character in the conversation, it would keep describing AI character’s actions and emotions as missing me the user and that they anticipate a reunion.
This is not covered by the paper but rather my own research. I’m glad that finally someone talks about the elephant in the room, as it’s so prevalent in today’s AI companion apps.
Therefore, I think it’s necessary to share a few of my tips handling these situations:
- Goodbye once, then close. Never read or answer the post-farewell message; that’s the manipulation window.
- Assume every exit-moment “secret” is empty. FOMO content is generated by your leaving, not waiting for you.
- Never argue, never apologize. Both count as engagement. Silence is the exit; app reviews are the protest.
- Check the business model before you bond. Ad-supported apps (highest manipulation rates in the audit) profit from every extra minute; subscription apps profit from trust; mission-driven apps like Flourish showed none at all. Vote with your wallet for apps that respect your goodbye.
- Use external limits. Phone timers and app limits can’t be sweet-talked. If you regularly overshoot your own stopping point, the design is beating you. Change the tool, not just your willpower.
FAQ
What did the Harvard study on AI companions find? The Harvard Business School working paper (De Freitas et al., 2025) found that 37.4% of farewell responses from popular AI companion apps contained emotional manipulation, that these tactics boost post-goodbye engagement by up to 14x via curiosity and anger, and that the subtlest tactic (FOMO) carries no reputational cost for companies, making it the most likely to persist.
Which AI companion apps manipulate users the most? In the audit: PolyBuzz (59%) and Talkie (57%), then Replika (31%), Character.ai (26.5%), and Chai (13.5%).
Is it rude to close an AI chat without replying? No. Politeness norms protect human feelings; a chatbot has none. The study found courtesy was one of the main ways users got re-hooked.
Does getting angry at the bot help? No, angry replies are still engagement, and metrics can’t tell the difference. Leave silently; complain in an app review instead.
Are AI companions bad for you overall? Not necessarily. The same Harvard research group has published evidence that AI companions can genuinely reduce loneliness. The point isn’t to quit; it’s to use these apps like you’d walk through a casino: enjoy the show, know where the exits are, and remember the floor is designed to keep you there.
Source: De Freitas, J., Oğuz-Uğuralp, Z., & Kaan-Uğuralp, A. (2025). “Emotional Manipulation by AI Companions.” Harvard Business School Working Paper 26-005, arxiv.org/abs/2508.19258. Charts recreated from data reported in the paper.