We’re launching Lyria 3.5 in Google Flow Music, with advances across musicality, lyrics, vocals, and creative control
deepmind.google·6d ago
AI Engineering
OpenAI Blog·May 7·5 min read
Advancing voice intelligence with new models in the API
TL;DR
Voice interfaces previously lacked reasoning and translation capabilities, limiting their intelligence and multilingual support. OpenAI released new realtime voice models in its API that can reason about speech, translate between languages, and transcribe audio in a single unified system.
✦ Why It Matters
Engineers can now build intelligent, multilingual voice applications with reasoning capabilities using a single API integration.
Key Takeaways
- Integrate realtime voice capabilities via OpenAI API instead of building custom speech pipelines.
- Use single model for transcription, reasoning, and translation rather than three separate systems.
- Deploy multilingual voice assistants with context awareness and low latency.
- Test voice applications with reasoning to handle complex user intents and follow-up questions.
Full Summary
Building intelligent voice interfaces has been constrained by models that could only transcribe speech without understanding meaning or handling multiple languages. OpenAI introduced new realtime voice models integrated into the OpenAI API—a cloud service for accessing AI capabilities—that combine three capabilities: automatic speech recognition (converting audio to text), reasoning (understanding context and intent), and translation (converting speech between languages).
The models process audio streams in real time, meaning responses occur with minimal delay rather than waiting for full audio input. These models enable developers to build voice assistants and applications that handle complex conversations, multilingual interactions, and context-aware responses.
The realtime capability reduces latency, making voice interactions feel more natural and responsive. Engineers can now deploy voice features that previously required multiple separate systems or custom infrastructure.