EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Hacker News·Sep 11·5 min read
We Replaced MMAP with Io_uring in Our Rust Query Engine. It Got Slower
TL;DR
Conviva replaced mmap with Io_uring in their Rust-based query engine, expecting performance gains, but instead experienced slower query response times under heavy loads.
✦ Why It Matters
Evaluate your I/O strategies under load to ensure they meet performance expectations in production environments.
Key Takeaways
- Monitor page faults to identify performance bottlenecks.
- Consider the implications of shared state in concurrent systems.
- Test I/O strategies in controlled environments before production.
- Optimize data handling to reduce unnecessary memory management overhead.
Full Summary
At Conviva, mmap was initially used for efficient reading of large Arrow IPC files in a Rust-based query engine. However, under heavy concurrent loads, mmap caused significant latency issues due to page cache thrashing and lock contention.
The team attempted to replace mmap with io_uring to bypass the page cache and improve performance. Initial tests showed a reduction in major faults but resulted in a 60% increase in total query time compared to mmap.
The architecture, particularly the Batch Materialization Layer, introduced complexities that hindered performance gains. Despite the promise of io_uring, the implementation did not meet expectations, highlighting the challenges of optimizing I/O in high-concurrency environments.