We’re launching Lyria 3.5 in Google Flow Music, with advances across musicality, lyrics, vocals, and creative control
deepmind.google·6d ago
AI Engineering
OpenAI Blog·Sep 30·5 min read
Launching Sora responsibly
TL;DR
Video generation models like Sora create novel safety risks—deepfakes, misinformation, and harmful content—that existing safeguards don't address. OpenAI built Sora 2 and the Sora app with safety mechanisms embedded from the start, including content filters and usage policies.
✦ Why It Matters
Engineers building generative video systems can adopt safety-first architecture patterns and policy enforcement mechanisms from Sora's deployment approach.
Key Takeaways
- Embed safety mechanisms during model design, not as post-hoc additions to production systems.
- Define concrete, measurable safeguards—content filters, detection systems—rather than relying on policy statements alone.
- Identify domain-specific risks: video generation enables deepfakes and misinformation at scale.
- Restrict high-risk use cases explicitly and monitor for policy violations in deployed applications.
Full Summary
Sora represents a state-of-the-art video generation model capable of creating realistic video content from text prompts, introducing safety challenges distinct from image or text systems. Unlike previous AI tools, video models enable creation of synthetic media that can convincingly depict events, raising risks around misinformation, non-consensual deepfakes, and harmful impersonation.
OpenAI developed Sora 2 and launched the Sora app with safety mechanisms integrated into the foundation rather than added afterward. The approach includes content policy enforcement, detection systems to flag policy violations, and restrictions on certain use cases.
The company emphasizes concrete protections over abstract commitments, though the blog post does not quantify detection rates, false positive rates, or user impact metrics. This responsible deployment model prioritizes preventing misuse while enabling legitimate creative applications.