We’re launching Lyria 3.5 in Google Flow Music, with advances across musicality, lyrics, vocals, and creative control
deepmind.google·6d ago
AI Engineering
OpenAI Blog·Oct 27·5 min read
Strengthening ChatGPT’s responses in sensitive conversations
TL;DR
ChatGPT previously struggled to recognize mental health distress and respond appropriately in sensitive conversations, risking harmful outputs. OpenAI partnered with 170+ mental health experts to enhance distress recognition, empathetic response generation, and real-world support guidance through specialized training.
✦ Why It Matters
Engineers can apply expert-in-the-loop training and RLHF to reduce unsafe outputs in safety-critical AI applications.
Key Takeaways
- Collaborate with domain experts to identify failure modes before designing safety interventions.
- Use RLHF and fine-tuning to encode specialized behavioral patterns into LLMs.
- Measure safety improvements quantitatively; 80% reduction in unsafe outputs validates the approach.
- Design guardrails that guide users toward professional resources, not replace expert help.
Full Summary
ChatGPT and similar large language models (LLMs—AI systems trained on vast text data) lacked specialized capability to handle mental health crises safely, potentially providing inadequate or harmful guidance when users disclosed distress. OpenAI engaged 170+ mental health professionals to identify failure modes and design improvements to ChatGPT's response patterns.
The collaboration focused on three capabilities: detecting linguistic and contextual signals of psychological distress, generating empathetic and non-judgmental replies, and directing users toward professional resources like crisis hotlines. The approach likely involved fine-tuning (adjusting model weights on curated examples) and reinforcement learning from human feedback (RLHF—ranking outputs by expert preference).
Results showed unsafe response rates dropped by up to 80% in sensitive conversation scenarios, indicating substantial risk reduction. This work demonstrates how domain expert collaboration can systematically improve AI safety in high-stakes applications.