Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·Jul 24·5 min read
Multimodal Pretraining for Generalizable EEG Representation Learning
TL;DR
A multimodal EEG foundation model was developed to enhance seizure detection across various datasets, achieving state-of-the-art performance with an AUROC of 0.878. This model supports robust generalization and interpretable seizure localization without requiring labeled data.
✦ Why It Matters
Consider implementing this multimodal EEG model to improve seizure detection capabilities in your projects.
Key Takeaways
- Explore the Mamba architecture for raw signal encoding.
- Utilize Vision Transformers for time-frequency data analysis.
- Implement masked modeling for effective pretraining.
- Adapt the model for patient-independent seizure detection.
Full Summary
Electroencephalography (EEG) models for epilepsy often struggle with generalization across different datasets and tasks. To address this, a multimodal EEG foundation model was created, integrating a raw signal encoder based on the Mamba architecture, a Vision Transformer (ViT)-style encoder for time-frequency data, and a lightweight text encoder.
The pretraining utilized innovative techniques like masked modeling and cross-view contrastive alignment to generate seizure-relevant representations without labeled data. The model was fine-tuned on the CHB-MIT seizure detection benchmark, achieving an AUROC of 0.874 for a single model and 0.878 for an ensemble variant.
Additionally, performance was evaluated using a leave-one-subject-out protocol, revealing a mean balanced accuracy of 0.558 across 19 subjects, highlighting the challenges of patient-independent seizure detection. This multimodal approach not only enhances seizure detection but also allows for easy adaptation to new scenarios.