We’re launching Lyria 3.5 in Google Flow Music, with advances across musicality, lyrics, vocals, and creative control
deepmind.google·6d ago
AI Engineering
OpenAI Blog·May 7·5 min read
Introducing Trusted Contact in ChatGPT
TL;DR
AI chatbots lack mechanisms to intervene when users express severe self-harm intent, creating a safety gap in mental health crisis detection. OpenAI built Trusted Contact, an optional feature that detects self-harm concerns in ChatGPT conversations and notifies a pre-selected trusted person.
✦ Why It Matters
Engineers can design safety features that detect crises while preserving user agency through consent-based notification systems.
Key Takeaways
- Implement consent-based notification systems for mental health crises rather than automated enforcement.
- Design safety features that empower trusted humans to intervene, not replace human judgment.
- Require explicit user authorization and contact verification to prevent misuse of notification systems.
- Balance privacy protection with crisis detection by letting users control feature activation and recipient selection.
Full Summary
ChatGPT users experiencing mental health crises have limited safety mechanisms within the platform itself. OpenAI developed Trusted Contact as an opt-in safety feature integrated into ChatGPT that monitors conversation content for indicators of serious self-harm risk.
When the system detects concerning patterns, it sends a notification to a contact person the user has pre-designated and authorized. This approach balances privacy (users control who is notified and whether the feature is active) with intervention capability (trusted contacts can reach out proactively).
The feature represents a shift toward human-in-the-loop crisis response rather than automated account suspension or content blocking. Implementation requires explicit user consent and contact verification to prevent misuse.
This addresses a critical gap: AI systems can identify risk signals but lack authority or capability to provide direct help.