EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Lobsters·Sep 9·5 min read
Every Millisecond Counts
TL;DR
Over four months, incremental optimizations were made to reduce a ClickHouse query's execution time from over a minute to under one second. These changes focused on improving performance for high-impact queries crucial for client experience.
✦ Why It Matters
Evaluate your database's handling of mutable data to identify potential performance improvements.
Key Takeaways
- Reassess partitioning strategies to optimize data reads.
- Precompute values at the application level to reduce query complexity.
- Eliminate unnecessary joins to improve execution speed.
- Monitor and adjust merge settings for better data management.
How It Works
The team implemented a new partitioning strategy based on client IDs rather than time, which improved insert performance and allowed for better parallelism during query execution. They also adjusted the ORDER BY clause to prioritize columns that would reduce the amount of data read, thus speeding up the query process.