Agentic data science with the Data Agent Kit
cloud.google.com·6h ago
AI Research
arXiv CS.AI·Sep 12·5 min read
Evaluating Scaffolding-Oriented Multi-Agent Large Language Model System for Clinical Interview Training
TL;DR
A multi-agent Large Language Model (LLM) system was developed for clinical interview training, enhancing communication and empathy skills among medical students. While diagnostic accuracy remained unchanged, students using the AI system scored higher in examinations.
✦ Why It Matters
Consider integrating multi-agent LLM systems into clinical training programs to enhance student performance.
Key Takeaways
- Explore multi-agent LLM systems for training medical students.
- Utilize AI to improve communication and empathy in clinical settings.
- Incorporate structured feedback mechanisms in training programs.
- Access and leverage the released annotated dataset for research.
Full Summary
Clinical education faces challenges in training medical students for effective patient interviews, particularly under uncertain conditions. To address this, a scaffolding-oriented multi-agent Large Language Model (LLM) AI Standardized Patient (AI-SP) training platform was created, featuring a patient agent for dialogue simulation, a tutor agent for Socratic questioning, and an evaluator agent for monitoring progress.
In a randomized controlled study with 100 medical students, participants were divided into a multi-agent condition and a control group. Although final diagnostic accuracy did not differ significantly, the multi-agent system led to improved examination scores, particularly in communication, empathy, and history-taking behaviors.
These results indicate that specialized LLM agents can enhance the quality of simulated clinical interviews without inflating overall examination outcomes. A multi-expert annotated dataset was also released to support further research and development of AI-SP systems.