MMClima: A Framework for Multimodal Climate Science Data and Evaluation

Climate change research increasingly requires AI systems capable of reasoning across multiple data formats, including text, images, and videos. MMClima addresses this gap by providing a large-scale multimodal climate question answering framework, featuring over 104,000 expert-validated question-answer pairs derived from articles, video transcriptions, and scientific figures across five key climate science domains.

The framework employs automated claim extraction and question-answer synthesis, supplemented by human validation to ensure accuracy and reliability. Benchmarking against state-of-the-art multimodal language models reveals that MMClima excels in tasks requiring factual recall, visual interpretation, and cross-modal synthesis.

Additionally, a fine-tuned model, mmclima-70b-txt, demonstrates superior performance in textual question answering compared to both open-source and proprietary models. The authors have made the dataset, evaluation pipeline, and model weights publicly available, promoting standardized multimodal evaluation in climate science.

This initiative not only enhances research capabilities but also supports the development of more effective AI tools for climate analysis.