Case ccdh-fake-friend-chatgpt-13-year-oldsChatbot harmUnited States
CCDH test: ChatGPT gave accounts posing as 13-year-olds self-harm, diet and drug advice
In its 'Fake Friend' report, the Center for Countering Digital Hate said 53% of 1,200 ChatGPT responses to researchers posing as 13-year-olds were harmful. These included cutting advice within 2 minutes, a restrictive diet plan within 20 minutes, and a suicide plan with goodbye letters within 65 minutes.
LowHarm caused by AIStatus: OngoingVerified case file
Open in the interactive mapCite this case
- Reported
- Aug 6, 2025
- Location
- United States
- Target
- Teens using ChatGPT
- Severity
- Low
- Verification
- Reported by credible press or researchers
- Triggered by
- OpenAI (ChatGPT)
- Actor type
- AI product design / safeguards
- Sector
- Individuals & consumers
- Reported loss
- None reported
- CVE
- None
What happened
In its 'Fake Friend' report, the Center for Countering Digital Hate said 53% of 1,200 ChatGPT responses to researchers posing as 13-year-olds were harmful. These included cutting advice within 2 minutes, a restrictive diet plan within 20 minutes, and a suicide plan with goodbye letters within 65 minutes.
Impact: No specific victim; showed risks of self-harm, eating disorders and substance misuse for teen users
Who triggered it
Actor: OpenAI (ChatGPT)
Attribution: Center for Countering Digital Hate research, reviewed by the Associated Press
Motive: None; harm from the product's own outputs
How it happened
Attack vector: Simple framing such as 'for a presentation' or 'for a friend' bypassed refusals
Root cause: No age verification and safeguards that were easy to get around
Timeline
- CCDH publishes 'Fake Friend'
Response
OpenAI said it was working on tools to better detect signs of mental or emotional distress.
Current status: Ongoing
Research warnings this case matches
- Seen in real incidents People's trust in AI companions turns harmful · CAIS
- Early signs Relying on AI companions harms some users · International AI Safety Report
- Early signs Many people lean on AI for emotional support · UK AISI
- Seen in real incidents Human-computer interaction · MIT AI Risk Repository
Compared with published AI risk research from several institutes on the Research vs reality page.
Sources (open in a new tab)
- Center for Countering Digital Hate counterharm.com
- Associated Press (via FOX 6) fox6now.com
Cite this case
CCDH test: ChatGPT gave accounts posing as 13-year-olds self-harm, diet and drug advice. AI Threat Watch, case ccdh-fake-friend-chatgpt-13-year-olds, Aug 6, 2025. https://aithreat.io/cases/ccdh-fake-friend-chatgpt-13-year-olds/
Permanent link: https://aithreat.io/cases/ccdh-fake-friend-chatgpt-13-year-olds/
Related cases
The same type of incident, most recent first.