NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·3h ago
AI Research
arXiv CS.AI·Jun 9·5 min read
Stress-testing medical large language models reveals latent safety pathology beyond benchmark accuracy
TL;DR
Large language models (LLMs) used in healthcare may overlook critical safety issues despite high benchmark accuracy. The AI-MASLD framework was developed to stress-test these models using 240 clinical cases and various performance metrics.
✦ Why It Matters
Engineers and researchers should incorporate stress-testing frameworks to ensure the safety and reliability of AI models in healthcare.
Key Takeaways
- Implement stress-testing frameworks for AI models in healthcare.
- Evaluate models using multiple performance metrics beyond accuracy.
- Recognize the limitations of supervised fine-tuning on model stability.
- Consider open-weight models for improved safety performance.
Full Summary
As large language models (LLMs) are increasingly integrated into clinical settings, their reliance on benchmark accuracy raises concerns about undetected safety issues. To address this, AI-MASLD, a stress-audit framework, was created, inspired by metabolic stress testing in hepatology.
The framework evaluated seven LLMs using 240 clinical cases and three performance metrics: metabolic index (MI), perturbation flip rate (PFR), and counterfactual fairness index (CFI). While models performed well under ideal conditions, stress testing revealed two distinct response types, with quantized models showing 'pseudonormalization'—low flip rates masking functional failures.
Additionally, supervised fine-tuning negatively impacted logical stability and fairness. An open-weight model outperformed proprietary models across all safety metrics.
These findings suggest that narrative stress auditing is essential for a comprehensive evaluation of clinical LLMs.