We’re launching Lyria 3.5 in Google Flow Music, with advances across musicality, lyrics, vocals, and creative control
deepmind.google·6d ago
AI Engineering
OpenAI Blog·Sep 29·5 min read
Combating online child sexual exploitation & abuse
TL;DR
Online platforms face a critical gap: AI systems can be misused to generate, distribute, or facilitate child sexual abuse material (CSAM). OpenAI implemented strict usage policies, detection tools, and industry reporting mechanisms to identify and block such content.
✦ Why It Matters
Engineers can implement detection systems and reporting pipelines to prevent AI misuse for child exploitation at scale.
Key Takeaways
- Implement explicit usage policies prohibiting CSAM-related requests in model terms of service.
- Build automated detection systems to identify exploitation-related prompts before content generation.
- Establish mandatory reporting pipelines to NCMEC and law enforcement for serious violations.
- Create cross-industry partnerships to share detection signals and enforcement best practices.
Full Summary
Child sexual exploitation and abuse online represents a severe harm that AI systems can inadvertently enable through content generation or distribution facilitation. OpenAI developed a multi-layered defense combining usage policies that explicitly prohibit CSAM-related requests, advanced detection tools that identify attempts to misuse AI models for exploitation, and mandatory reporting protocols integrated with the National Center for Missing and Exploited Children (NCMEC).
The approach combines automated detection with human review and cross-industry collaboration to catch violations at multiple stages. OpenAI established clear enforcement mechanisms including account suspension and law enforcement referrals for serious cases.
This framework demonstrates that AI safety requires proactive technical controls, clear policy boundaries, and institutional partnerships rather than reactive responses alone.