Saving another 100TB of RAM with math (and Rust)

TL;DR

Cloudflare optimized its Pingora Backend Router by refining its consistent hashing algorithm, resulting in a remarkable reduction of over 100TB of RAM usage globally.

✦ Why It Matters

Evaluate your own systems for potential memory optimizations using mathematical approaches.

Key Takeaways

Investigate memory usage in your applications regularly.
Consider using consistent hashing for load distribution.
Optimize data structures to reduce memory footprint.
Implement gradual rollouts for system changes to minimize risk.

How It Works

The optimization involved changing the data structure used for storing hashes in the consistent hashing algorithm. By reducing the size of the index field and using a raw byte array, memory usage was cut by 25%.

Additionally, increasing the number of hashes per server improved workload distribution, lowering the coefficient of variation from 99% to 8%.

Saving another 100TB of RAM with math (and Rust)

Reimagining service delivery in the agentic era with Google Public Sector

Announcing Native BM25 Ranking in AlloyDB and Cloud SQL

Changing the game: How Google uses agentic AI to secure hundreds of millions of lines of code

The Download: AI’s extinction risk and bioweapons threat

Reimagining service delivery in the agentic era with Google Public Sector

Announcing Native BM25 Ranking in AlloyDB and Cloud SQL

Changing the game: How Google uses agentic AI to secure hundreds of millions of lines of code

The Download: AI’s extinction risk and bioweapons threat