Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·May 21·5 min read
Synergistic Foundation Models for Semi-Supervised Fetal Cardiac Ultrasound Analysis: SAM-Med2D Boundary Refinement and DINOv3 Semantic Enhancement
TL;DR
Doctors struggle to accurately segment (outline) and classify heart structures in fetal ultrasound images, especially with limited labeled training data. Researchers built a semi-supervised framework combining EchoCare (a multi-task cardiac model), SAM-Med2D (a boundary-refinement tool), and DINOv3 (a self-supervised feature extractor) with view-specific masking and two-stage training—first consolidating segmentation via EMA (exponential moving average), then fine-tuning classification.
✦ Why It Matters
Engineers can apply this semi-supervised pattern—combining foundation models with domain-specific refinement—to medical imaging tasks where labeled data is scarce or expensive.
Key Takeaways
- Combine SAM-Med2D with DINOv3 to improve segmentation and classification in medical imaging with limited labels.
- Implement two-stage optimization: EMA consolidation phase followed by classification fine-tuning for better convergence.
- Apply view-specific hard masking to focus model learning on clinically relevant image regions.
- Use pseudo-labels from self-supervised models to reduce manual annotation requirements in ultrasound analysis.
Full Summary
Fetal cardiac ultrasound analysis demands simultaneous segmentation (identifying anatomical structures) and classification (determining cardiac view type), but obtaining fully labeled medical imaging datasets is expensive and time-consuming. The authors developed a semi-supervised framework—one that learns from both labeled and unlabeled data—built on EchoCare, a multi-task backbone model designed for cardiac ultrasound.
They integrated SAM-Med2D, a medical imaging adaptation of the Segment Anything Model that refines object boundaries, and DINOv3, a self-supervised learning model that extracts semantic features to improve pseudo-labels (automatically generated training labels). The method employs view-specific hard masking to focus learning on relevant image regions and uses a two-stage optimization: an EMA (Exponential Moving Average) phase to stabilize segmentation, followed by classification fine-tuning.
This synergistic combination of foundation models reduces annotation burden while maintaining diagnostic accuracy for clinical fetal cardiac assessment.