EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Hacker News·Sep 11·5 min read
118M Queries per Second on Neki
TL;DR
Neki achieved a remarkable benchmark of 118 million queries per second (QPS) across 512 shards, significantly surpassing initial goals. This performance was tested using a simple point select query methodology.
✦ Why It Matters
Consider testing your own systems against high query loads to identify performance limits.
Key Takeaways
- Test your system's scalability with increased shard counts.
- Focus on optimizing read-only workloads for better performance.
- Monitor latency and error rates during high-load scenarios.
- Prepare for future engineering challenges by analyzing current benchmarks.
Full Summary
Neki, a new platform, was tested for its query performance, aiming to handle 1 million queries per second (QPS). The testing quickly escalated to 512 shards, achieving a peak of 118,538,803 QPS while sustaining this rate for 16 minutes.
Each shard operated with a single Postgres primary instance, and the workload was limited to point-select queries, meaning each query fetched one row by its primary key. The system demonstrated impressive efficiency, with a p99 latency of 6.06ms at the router and 13.95ms at the client.
Notably, the test recorded only 67 errors per second, translating to one error for every 1.8 million queries. Future articles will delve into the engineering challenges faced during this achievement.