We’re launching Lyria 3.5 in Google Flow Music, with advances across musicality, lyrics, vocals, and creative control
deepmind.google·6d ago
AI Engineering
Hacker News·22d ago·5 min read
Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor
TL;DR
Apple's new SpeechAnalyzer API outperforms its predecessor and Whisper models in accuracy and speed. Tested on 5,559 utterances, it achieved the lowest word error rate.
✦ Why It Matters
Developers should migrate to the SpeechAnalyzer API for improved accuracy in on-device speech recognition applications.
Key Takeaways
- Migrate to SpeechAnalyzer for better accuracy in speech recognition.
- Utilize the LibriSpeech dataset for testing speech models.
- Benchmark performance on Apple M2 Pro for consistent results.
- Consider the speed advantage of SpeechAnalyzer in real-time applications.
Full Summary
Apple introduced the SpeechAnalyzer API to replace the SFSpeechRecognizer without providing accuracy metrics, prompting a need for independent evaluation. A benchmark was conducted using 5,559 standard test utterances from the LibriSpeech dataset, which includes both clean and noisy speech samples.
The tests revealed that SpeechAnalyzer surpassed all Whisper models, including Whisper Small, achieving a lower word error rate (WER) and running approximately three times faster. In contrast, the previous SFSpeechRecognizer performed poorly, even lagging behind Whisper Tiny, a smaller model.
All tests were conducted on an Apple M2 Pro with 32GB of RAM, ensuring consistent conditions. These findings indicate that SpeechAnalyzer is the most accurate on-device speech recognition engine available, making it a compelling choice for developers.