Agentic data science with the Data Agent Kit
cloud.google.com·8h ago
AI Research
arXiv CS.AI·Sep 10·5 min read
False positive bias in AI-powered speech-based cognitive screening for multilingual English speakers in the UK
TL;DR
AI models for speech-based cognitive screening show promise but exhibit significant false positive bias for multilingual English speakers in the UK, particularly among ethnic minorities. Multilingual individuals were 2.5 times more likely to be misclassified as cognitively impaired compared to monolingual speakers.
✦ Why It Matters
Develop and validate AI models that account for multilingualism to reduce false positives in cognitive screening.
Key Takeaways
- Evaluate AI models for bias in multilingual contexts.
- Collect diverse speech data to improve model accuracy.
- Consider cultural and linguistic factors in cognitive assessments.
- Implement bias mitigation strategies in AI development.
Full Summary
Conversational speech can indicate early cognitive decline, such as dementia and mild cognitive impairment (MCI). This study involved 1,395 participants, including both monolingual English speakers and multilingual speakers from diverse backgrounds in the UK.
Over 263 hours of speech data were collected using the CognoMemory agent. While automatic speech recognition (ASR) models like Whisper and Wav2Vec 2.0 performed similarly across groups, downstream AI models showed significant disparities.
Multilingual speakers faced false positive rates of 28 to 37%, compared to 12 to 16% for monolingual speakers, indicating they were about 2.5 times more likely to receive incorrect impairment labels. These biases were exacerbated when models were trained on the DementiaBank dataset.
This research highlights the need to address biases in AI cognitive screening tools to ensure equitable healthcare for multilingual populations.