We’re launching Lyria 3.5 in Google Flow Music, with advances across musicality, lyrics, vocals, and creative control
deepmind.google·6d ago
AI Engineering
OpenAI Blog·Jan 22·8 min read
Scaling PostgreSQL to power 800 million ChatGPT users
TL;DR
OpenAI needed to handle massive query volume from 800 million ChatGPT users hitting a single PostgreSQL database, which would overwhelm traditional setups. They implemented read replicas (copies of the database for queries), in-memory caching, rate limiting (throttling requests), and workload isolation (separating different request types).
✦ Why It Matters
Engineers can scale existing databases beyond apparent limits using replicas, caching, and isolation instead of costly rewrites.
Key Takeaways
- Deploy read replicas to distribute query load across multiple database copies.
- Implement caching layers to reduce repeated database hits for hot data.
- Use rate limiting to prevent request spikes from overwhelming the database.
- Isolate workloads by request type to protect critical queries from interference.
Full Summary
OpenAI faced a critical scaling challenge: a single PostgreSQL database (a relational database system storing structured data) receiving millions of concurrent requests from 800 million users. Rather than migrating to a different database system, they optimized PostgreSQL through four complementary techniques.
Read replicas distribute read-only queries across multiple database copies, reducing load on the primary instance. Caching layers store frequently accessed data in memory to avoid repeated database hits.
Rate limiting controls request volume by rejecting or delaying excess queries. Workload isolation separates different request types into dedicated resources, preventing high-priority queries from being blocked by low-priority ones.
Together, these techniques enabled PostgreSQL to sustain millions of queries per second—a throughput typically associated with specialized distributed systems—while maintaining consistency and reliability.