Agentic data science with the Data Agent Kit
cloud.google.com·6h ago
AI Research
arXiv CS.AI·Sep 10·5 min read
Forward-Free LLM Depth Pruning via Weight Redundancy
TL;DR
Weight-Redundancy Pruning (WRP) is a novel forward-free method for depth pruning in large language models (LLMs) that effectively reduces inference costs by identifying and removing redundant Transformer blocks. WRP outperforms existing methods, achieving results comparable to activation-based techniques without requiring calibration data.
✦ Why It Matters
Implement WRP to enhance your LLM's efficiency without the overhead of calibration data.
Key Takeaways
- Explore WRP for efficient LLM inference cost reduction.
- Utilize WRP to eliminate redundant Transformer blocks.
- Consider WRP for various model families and tasks.
- Leverage WRP's performance close to activation-based methods.
How It Works
WRP operates by evaluating the redundancy between layers using the model's checkpoint weights. It compares the attention outputs and MLP down-projection weights across different layers to create a similarity matrix.
This matrix helps in grouping layers and selecting which Transformer blocks can be pruned without significant loss in performance.