Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·7d ago·5 min read
DriftTTS: Few-Step Text-to-Speech Without Distillation via Distribution-Matching Drift
TL;DR
DriftTTS introduces a novel few-step text-to-speech model that generates mel-spectrograms without relying on distillation or generative teachers. It achieves competitive performance with a mean opinion score of 4.18 in blind listening tests.
✦ Why It Matters
Explore DriftTTS for efficient text-to-speech synthesis in your next project without needing extensive pre-training.
Key Takeaways
- Implement DriftTTS for efficient speech synthesis in your applications.
- Utilize the distribution-matching drift objective for improved model training.
- Achieve competitive audio quality without pre-trained generative models.
- Experiment with on-policy rollout techniques for better intermediate state learning.
How It Works
DriftTTS operates by generating mel-spectrograms directly from text using a distribution-matching drift objective. This approach allows the model to learn the distribution of mel-spectrograms without needing a generative teacher.
The frozen masked autoencoder serves as a feature extractor, while the decoder is trained on its own outputs through on-policy rollouts, enabling it to refine its predictions based on intermediate states.