We’re launching Lyria 3.5 in Google Flow Music, with advances across musicality, lyrics, vocals, and creative control
deepmind.google·6d ago
AI Engineering
Hacker News·15d ago·5 min read
Lossless model compression experiment: GLM-5.2 in 25% less memory
TL;DR
A new lossless compression method for the GLM-5.2 model reduces memory usage by 25% without losing data fidelity. By leveraging a K15 charged-format accounting system, the model efficiently encodes repetitive weight exponents.
✦ Why It Matters
Engineers can implement the K15 compression method to reduce memory usage in large AI models today.
Key Takeaways
- Explore K15 accounting for efficient model compression.
- Validate tensor reconstruction to ensure lossless compression.
- Consider dense prototypes for improved performance metrics.
- Review existing methods like ZipNN for additional insights.
How It Works
The K15 accounting method optimizes memory usage by replacing 9-bit sign-and-exponent symbols with a 4-bit code, significantly reducing the size of BF16 tensors. This method leverages the repetitive nature of exponents in trained models, allowing for efficient encoding and reconstruction of weights.