EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Hugging Face Blog·Aug 28·5 min read
The Open ASR Leaderboard Adds Its First Global South Language
TL;DR
The Open ASR Leaderboard has introduced its first global South language, Hindi, with two new evaluation sets aimed at addressing bias in automated speech recognition (ASR). This addition highlights the need for more inclusive benchmarks that reflect diverse speaker attributes.
✦ Why It Matters
Incorporate diverse language datasets in your ASR projects to enhance inclusivity and performance.
Key Takeaways
- Utilize Monsoon datasets for training ASR models.
- Analyze performance disparities across demographics using new evaluation metrics.
- Implement lattice structures for handling orthographic variations in Hindi.
- Focus on spontaneous speech data to improve model robustness.
How It Works
The Monsoon datasets were created by collecting spontaneous speech from diverse speakers across India, ensuring representation from various regions and demographics. Each audio clip is accompanied by extensive metadata, including speaker attributes and geographic information, allowing for detailed analysis of ASR performance across different groups.
The Hindi dataset's lattice structure enables the acceptance of multiple correct spellings, addressing the challenges of Hindi orthography.