Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·Jun 9·5 min read
Automatic Extraction of Structured Information from Brain MRI Reports Using an Open-Weight Large Language Model
TL;DR
Free-text radiology reports often hinder large-scale research due to the difficulty of extracting structured data. An open-weight large language model (LLM) was applied to analyze 947 Dutch brain MRI reports, focusing on extracting thirty specific variables.
✦ Why It Matters
Engineers and researchers can leverage LLMs for efficient data extraction from unstructured medical texts, enhancing research capabilities.
Key Takeaways
- Explore LLMs for automating data extraction in medical research.
- Consider inter-rater reliability when annotating data for accuracy.
- Utilize structured data to enhance research insights and patient outcomes.
- Analyze the performance of LLMs on specific medical report types.
Full Summary
Radiology reports, particularly those related to brain MRI scans, are typically unstructured, making it challenging to extract useful data for research. This study utilized an open-weight large language model (LLM) to automatically extract structured information from 947 neuroradiology reports written by consultant radiologists between 2016 and 2021.
Medical students annotated thirty variables from these reports, and a subset of 100 reports was double-annotated to evaluate inter-rater reliability, which measures the consistency between different annotators. The results indicated that the LLM performed effectively in extracting relevant data, demonstrating its potential for enhancing research capabilities in medical fields.
This approach not only streamlines data extraction but also opens avenues for large-scale analysis of medical reports, which can lead to improved patient outcomes and research insights.