EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Lobsters·Aug 27·5 min read
Float Bloat: vector serialization gone wrong
TL;DR
Bonsai has identified a critical issue in vector serialization that affects millions of implementations, where float32 vectors are incorrectly cast to float64, leading to unnecessary data bloat. This misstep results in over 20 Petabytes of excess storage costs globally.
✦ Why It Matters
Review your vector serialization practices to avoid unnecessary data bloat and reduce storage costs.
Key Takeaways
- Identify and fix float bloat in your vector serialization.
- Use float32 instead of float64 to save storage space.
- Implement explicit formatting to prevent automatic widening.
- Consider using binary or base64 formats for efficiency.
How It Works
The issue arises when float32 vectors are serialized, leading to automatic conversion to float64, which adds unnecessary precision. This conversion occurs in many programming languages when values leave typed arrays, resulting in increased storage and bandwidth usage.