Agentic data science with the Data Agent Kit
cloud.google.com·8h ago
AI Research
arXiv CS.AI·Sep 2·5 min read
Cleaner Speech, Weaker Generalization: Revisiting Pitt-Derived Benchmarks for Alzheimer's Disease Detection
TL;DR
Speech-enhanced datasets for Alzheimer's disease detection improve in-domain performance but weaken model generalization across different datasets. This study highlights the trade-offs involved in speech preprocessing.
✦ Why It Matters
Consider using unprocessed speech data for training models to enhance generalization in Alzheimer's detection.
Key Takeaways
- Evaluate the impact of speech preprocessing on model performance.
- Use unprocessed datasets to improve cross-domain generalization.
- Be cautious of class imbalance in enhanced datasets.
- Test models on diverse datasets to ensure robustness.
Full Summary
Alzheimer's disease (AD) detection using speech analysis often employs enhanced versions of the Pitt Corpus, which includes curated speech samples. This research investigates how speech preprocessing affects model performance, particularly focusing on deep learning models and large audio-language models (LALMs).
The study evaluates the quality of various datasets and the generalization capabilities of models trained on enhanced versus unprocessed data. Results indicate that while speech enhancement boosts performance within the same dataset, it significantly reduces robustness when models are tested on different datasets.
Specifically, matched enhancement between training and testing data helps but does not fully resolve the issue. Additionally, LALMs trained on enhanced datasets exhibit greater class imbalance and prediction shifts compared to those trained on raw data.
These findings suggest that cleaner speech datasets may not be the best choice for reliable real-world AD detection.