NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·May 26·5 min read
MuCRASP: Multimodal Chain-of-thought Reasoning aware Structured Pruning
TL;DR
Vision-language models (VLMs) struggle with high deployment costs due to their large parameter sizes, especially when using chain-of-thought (CoT) reasoning. MuCRASP, a structured pruning framework, was developed to maintain reasoning accuracy while reducing model size by targeting critical components.
✦ Why It Matters
Engineers can implement MuCRASP to optimize VLMs for cost-effective deployment without losing reasoning capabilities.
Key Takeaways
- Explore MuCRASP for efficient model pruning in VLMs.
- Focus on reasoning-critical components during model optimization.
- Consider activation distribution differences in multimodal models.
- Aim for high reasoning consistency even with significant pruning.
Full Summary
Vision-language models (VLMs) are increasingly used for complex tasks that require understanding both visual and textual information, but their large size makes them costly to deploy. Existing pruning methods, which reduce model size, often fail to maintain the accuracy of chain-of-thought (CoT) reasoning, which relies on specific tokens in the generation process.
MuCRASP was created to address this issue by focusing on reasoning-critical components and considering the differences in activation distributions between visual and textual modalities. The framework allows for structured pruning while ensuring cross-modal alignment and layer-wise sensitivity under a global parameter budget.
In experiments with four VLMs across three reasoning benchmarks, MuCRASP achieved an LLM-as-a-Judge score of 8.87 at 30% pruning, compared to 7.32 for the best baseline. It also maintained high reasoning consistency up to 50% pruning, demonstrating lower perplexity degradation than previous approaches.
These results suggest that MuCRASP can effectively reduce model size without sacrificing performance.