Accelerating the search for antimicrobial peptides with Google Antigravity
deepmind.google·2h ago
AI Research
arXiv CS.AI·1d ago·5 min read
RECAP: Calibrating Low-Bit Reasoning Models on the History They Read
TL;DR
RECAP is a novel post-training quantization method that enhances low-bit reasoning models by effectively calibrating them on their historical context. It significantly improves accuracy, achieving up to 15.8 percentage points better performance on specific reasoning tasks.
✦ Why It Matters
Implement RECAP in your low-bit reasoning models to significantly boost their accuracy on complex tasks.
Key Takeaways
- Explore implementing RECAP for improved model accuracy.
- Focus on historical context during model calibration.
- Evaluate performance on datasets like MATH500 and AIME-120.
- Consider the impact of quantization errors on reasoning tasks.
How It Works
RECAP employs dynamic-prefix calibration to distinguish between long historical context and short-block reconstruction. It regenerates prefix keys and values using the current quantizer, allowing for better supervision of cache errors.
This method ensures that historical dependencies are preserved, which is critical for accurate attention outputs in reasoning tasks.