Agentic data science with the Data Agent Kit
cloud.google.com·6h ago
AI Research
arXiv CS.AI·10d ago·5 min read
Counterfactual Auditing of Bias in Open-Source Large Language Models for Clinical Triage
TL;DR
A comparative audit of ten open-source large language models (LLMs) for pediatric Emergency Severity Index (ESI) prediction reveals significant variations in counterfactual bias. Notably, the fine-tuned Qwen2.5-7B model exhibited the lowest sensitivity to demographic changes, suggesting a potential framework for assessing fairness in clinical AI applications.
✦ Why It Matters
Evaluate your AI models for bias using counterfactual auditing to ensure fair clinical decision-making.
Key Takeaways
- Implement counterfactual auditing to assess model fairness.
- Consider fine-tuning models to reduce sensitivity to demographic changes.
- Evaluate larger models for potential bias before deployment.
- Use stratified analysis to uncover hidden failure patterns.
Full Summary
Emergency department triage is critical for prioritizing patient care, but biases related to demographics and socioeconomic factors can skew acuity assignments. This study evaluates ten open-source large language models (LLMs) for their performance in predicting the Emergency Severity Index (ESI) using counterfactual auditing.
By creating paired clinical vignettes that alter only one demographic or contextual variable, the researchers measured shifts in model predictions. Results showed that the fine-tuned Qwen2.5-7B model had a 5.27% rate of any shift and a mean absolute shift of 0.0534, significantly lower than the base model's 16.02% and 0.1706, respectively.
However, larger models and those trained in medical domains exhibited greater shifts. The findings highlight the importance of counterfactual auditing as a method for evaluating fairness risks in LLMs before they are deployed in clinical settings.