EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Hacker News·Jul 30·5 min read
Postgres Queues Actually Scale
TL;DR
Postgres can effectively handle high-scale queue workloads, achieving 30,000 workflow executions per second with proper optimizations. By utilizing locking mechanisms, contention among workers is minimized, allowing for efficient task processing.
✦ Why It Matters
Implement locking mechanisms in your Postgres queues to enhance scalability and performance today.
Key Takeaways
- Implement FOR UPDATE SKIP LOCKED to reduce contention.
- Use READ COMMITTED isolation for local queue limits.
- Optimize indexes to improve query performance.
- Monitor CPU usage and adjust indexing strategies accordingly.
How It Works
By using FOR UPDATE SKIP LOCKED, workers can lock and dequeue workflows without contention, allowing multiple workers to operate simultaneously. This method prevents multiple workers from attempting to dequeue the same workflow, significantly improving throughput.