NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·Jun 10·5 min read
What Do Deepfake Speech Detectors Actually Hear?
TL;DR
Deepfake speech detectors often lack transparency, making it unclear why certain audio samples are flagged. An audio-native explainability pipeline using Integrated Gradients was developed to identify decision evidence over time in three WavLM-based detectors.
✦ Why It Matters
Engineers can leverage these insights to enhance the transparency and effectiveness of deepfake detection systems.
Key Takeaways
- Implement Integrated Gradients for model explainability.
- Focus on specific audio cues for improved detection.
- Validate detector performance through causal masking.
- Explore different feature reliance in deepfake detection.
Full Summary
Deepfake speech detection is crucial for identifying manipulated audio, yet many detectors provide only a score without insight into their decision-making process. To address this, an audio-native explainability pipeline was created using Integrated Gradients, a technique that helps visualize which parts of an audio signal influence the detector's output.
This method was applied to three WavLM-based detectors: AASIST, CA-MHFA, and SLS, evaluated on the ASVspoof 5 dataset. The researchers annotated the most significant regions of the audio to provide semantic meaning to the cues used by each detector.
Results showed that while all detectors performed similarly, they relied on different features: AASIST focused on environmental sounds, CA-MHFA on phoneme artifacts, and SLS on word boundaries. Causal masking of these cues confirmed their importance, as performance dropped when key features were obscured.
These findings enhance understanding of deepfake detection mechanisms and improve future model development.