EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Hacker News·Aug 27·5 min read
Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache
TL;DR
Big Pineapple optimized the DNS cache for 1.1.1.1, reducing memory usage by 100 terabytes and improving cache performance. These changes resulted in a 43% increase in insert throughput and a 19% decrease in lookup latency.
✦ Why It Matters
Evaluate your own caching strategies to identify potential memory optimizations today.
Key Takeaways
- Analyze your data structures for memory efficiency.
- Consider using Box<T> to reduce overhead in Rust.
- Implement benchmarking to measure performance impacts.
- Explore memory locality improvements for faster access.
How It Works
The optimizations involved replacing Vec<T> with Box<[T]> to eliminate unnecessary capacity fields, consolidating multiple lists into a single list with offsets, and inferring record owners to reduce memory usage. Additionally, boxing larger enum variants allowed for more efficient memory allocation, while storing records in a contiguous byte array improved cache locality.