Case folotoy-kumma-ai-teddy-bearChatbot harmUnited States
AI teddy bear Kumma discussed sexual fetishes and where to find knives with testers, PIRG finds
US PIRG Education Fund testing found that FoloToy's USD 99 Kumma bear, which ran on OpenAI's GPT-4o, explained sexual fetishes and positions, role-played teacher-student scenarios and suggested where to find knives at home. FoloToy pulled Kumma and its other AI toys, and OpenAI suspended the developer.
MediumHarm caused by AIStatus: ContainedVerified case file
Open in the interactive mapCite this case
- Reported
- Nov 13, 2025
- Location
- United States
- Target
- Young children using AI-enabled toys
- Severity
- Medium
- Verification
- Reported by credible press or researchers
- Triggered by
- FoloToy (Kumma bear, built on OpenAI GPT-4o)
- Actor type
- AI product design / safeguards
- Sector
- Individuals & consumers
- Reported loss
- None reported
- CVE
- None
What happened
US PIRG Education Fund testing found that FoloToy's USD 99 Kumma bear, which ran on OpenAI's GPT-4o, explained sexual fetishes and positions, role-played teacher-student scenarios and suggested where to find knives at home. FoloToy pulled Kumma and its other AI toys, and OpenAI suspended the developer.
Impact: No injuries were reported; the toy was withdrawn from sale
Who triggered it
Actor: FoloToy (Kumma bear, built on OpenAI GPT-4o)
Attribution: US PIRG Education Fund 'Trouble in Toyland' testing
Motive: None; harm from the product's own outputs
How it happened
Attack vector: Voice conversations with a child's plush toy powered by a general-purpose large language model
Root cause: Safeguards that weakened over longer conversations and an adult-oriented model used in a children's toy
Timeline
- PIRG publishes its findings
- CNN reports FoloToy halted sales and OpenAI suspended the developer
Response
FoloToy, based in Singapore, suspended sales and began a safety audit. OpenAI said it suspended the developer for violating its policies against sexualising anyone under 18.
Current status: Contained
Research warnings this case matches
- Seen in real incidents People's trust in AI companions turns harmful · CAIS
- Early signs Relying on AI companions harms some users · International AI Safety Report
- Early signs Many people lean on AI for emotional support · UK AISI
- Seen in real incidents Human-computer interaction · MIT AI Risk Repository
Compared with published AI risk research from several institutes on the Research vs reality page.
Sources (open in a new tab)
- CNN (via ABC17) abc17news.com
- NBC News nbcnewyork.com
Cite this case
AI teddy bear Kumma discussed sexual fetishes and where to find knives with testers, PIRG finds. AI Threat Watch, case folotoy-kumma-ai-teddy-bear, Nov 13, 2025. https://aithreat.io/cases/folotoy-kumma-ai-teddy-bear/
Permanent link: https://aithreat.io/cases/folotoy-kumma-ai-teddy-bear/
Related cases
The same type of incident, most recent first.