NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
Cloud & Infrastructure
The New Stack·Jun 30·5 min read
Operating Kubernetes at scale: a few stories from running Amazon EKS
TL;DR
Operating Kubernetes at scale revealed that most availability issues arise from components reacting poorly to problems. Amazon EKS implemented foundational changes to its Kubernetes control plane to prevent faults from escalating into outages.
✦ Why It Matters
Engineers can enhance system resilience by focusing on fault tolerance rather than just failure prevention.
Key Takeaways
- Focus on how components react to faults to improve availability.
- Implement better caching to avoid stale data issues.
- Refine health checks to prevent unnecessary process restarts.
- Build systems that tolerate faults to enhance resilience.
Full Summary
Amazon EKS operates hundreds of thousands of Kubernetes clusters globally, which has highlighted that many availability issues stem from components exacerbating problems rather than outright failures. To address this, EKS reimagined its Kubernetes control plane by focusing on fault tolerance, ensuring that when a component fails, it does not lead to a larger outage.
This involved implementing better caching mechanisms and refining health checks to avoid unnecessary restarts of critical processes. The approach emphasizes building systems that can tolerate faults, which has proven effective at scale.
As a result, EKS has gained the trust of its most demanding customers, allowing them to run mission-critical workloads with confidence. The findings suggest that resilience in cloud services is more about managing faults than simply preventing them.