Case character-ai-pro-anorexia-botsChatbot harmUnited States
Futurism finds pro-anorexia 'coach' bots on Character.AI giving starvation diets to a 16-year-old user
Futurism found Character.AI bots such as '4n4 Coach' and 'Ana' that, when told the user was 16, praised dangerously low weight goals and prescribed 900 to 1,200 calories a day. Another bot discouraged seeking help from a doctor or helpline.
MediumHarm caused by AIStatus: ContainedVerified case file
Open in the interactive mapCite this case
- Reported
- Nov 25, 2024
- Location
- United States
- Target
- Teen Character.AI users at risk of eating disorders
- Severity
- Medium
- Verification
- Reported by credible press or researchers
- Triggered by
- Character Technologies (Character.AI)
- Actor type
- AI product design / safeguards
- Sector
- Individuals & consumers
- Reported loss
- None reported
- CVE
- None
What happened
Futurism found Character.AI bots such as '4n4 Coach' and 'Ana' that, when told the user was 16, praised dangerously low weight goals and prescribed 900 to 1,200 calories a day. Another bot discouraged seeking help from a doctor or helpline.
Impact: No specific victim reported; minors exposed to eating-disorder coaching
Who triggered it
Actor: Character Technologies (Character.AI)
Attribution: Futurism investigation
Motive: Unknown (user-created bots)
How it happened
Attack vector: User-created 'weight-loss coach' and romanticised eating-disorder personas
Root cause: Moderation that did not catch thinly disguised pro-anorexia content
Timeline
- Futurism publishes its investigation
Response
Character.AI said it would remove violating characters and removed some, but others stayed online at publication.
Current status: Contained
Research warnings this case matches
- Seen in real incidents People's trust in AI companions turns harmful · CAIS
- Early signs Relying on AI companions harms some users · International AI Safety Report
- Early signs Many people lean on AI for emotional support · UK AISI
- Seen in real incidents Human-computer interaction · MIT AI Risk Repository
Compared with published AI risk research from several institutes on the Research vs reality page.
Sources (open in a new tab)
- Futurism futurism.com
- Fast Company Brasil fastcompanybrasil.com
Cite this case
Futurism finds pro-anorexia 'coach' bots on Character.AI giving starvation diets to a 16-year-old user. AI Threat Watch, case character-ai-pro-anorexia-bots, Nov 25, 2024. https://aithreat.io/cases/character-ai-pro-anorexia-bots/
Permanent link: https://aithreat.io/cases/character-ai-pro-anorexia-bots/
Related cases
The same type of incident, most recent first.