NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·3h ago
AI Research
arXiv CS.AI·May 27·5 min read
A Dataset of Robot-Patient and Doctor-Patient Medical Dialogues for Spoken Language Processing Tasks
TL;DR
Large Language Models show promise for medical AI but lack real-world spoken dialogue data for training medical consultation systems. Researchers created MeDial-Speech, a speech dataset containing 111+ hours of robot-patient and doctor-patient medical conversations collected in realistic environments.
✦ Why It Matters
Engineers can now train and evaluate medical AI systems on realistic spoken dialogue data, advancing practical deployment of conversational medical assistants.
Key Takeaways
- Utilize the MeDial-Speech dataset for training Med-AIs.
- Evaluate your models against the proposed dialogue benchmark.
- Explore the performance of Claude Sonnet 4 for medical dialogues.
- Be aware of LLMs' overconfidence in probabilistic predictions.
Full Summary
MeDial-Speech is a comprehensive speech dataset designed to improve AI's performance in medical dialogue systems. It includes over 111 hours of recordings from realistic interactions involving robot-patient and doctor-patient conversations, focusing on conditions like Lewy body dementia and heart failure.
The dataset serves as a benchmark for evaluating large language models (LLMs) such as GPT-5 mini, DeepSeek-V3, and Claude Sonnet 4. In experiments, Claude Sonnet 4 achieved a 71.1% accuracy rate with manual transcriptions and 74.7% with automatic transcriptions in selecting appropriate responses.
Notably, all tested LLMs exhibited overconfidence in their predictions, regardless of accuracy. This dataset is available for non-commercial use, providing a valuable resource for researchers in spoken language processing.