Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·28d ago·5 min read
SteerDuplex: Steerable Duplex Speech Dialogue Models
TL;DR
SteerDuplex is a new full-duplex speech dialogue model that enhances steerability, allowing users to adjust tone, persona, and style based on instructions. It significantly improves interaction quality, achieving a 44.5 percentage point increase in audio-steering pass rates over existing models.
✦ Why It Matters
Explore SteerDuplex to enhance user interaction in your speech applications today.
Key Takeaways
- Implement SteerDuplex for improved conversational steerability.
- Utilize SteerBench for evaluating dialogue model performance.
- Adopt two-stage reinforcement learning for better response timing.
- Focus on both timing and completeness in dialogue evaluations.
How It Works
SteerDuplex utilizes a Moshi-based architecture and is fine-tuned on diverse dialogue datasets. It incorporates two-stage reinforcement learning, which combines direct interaction checks with semantic feedback from human judges to optimize response timing and continuity.
This approach allows the model to adapt its speech characteristics based on user instructions effectively.