EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Hacker News·25d ago·5 min read
How Uber Protects Against Retry Storms
TL;DR
Uber developed a context-aware retry mechanism to combat retry storms that can escalate service outages. This innovation improves error handling and protects user experience during service degradation.
✦ Why It Matters
Evaluate your retry strategies and consider implementing context-aware mechanisms to enhance service resilience.
Key Takeaways
- Adopt context-aware error handling to manage retries effectively.
- Analyze service dependencies to identify error ownership.
- Limit retries to services capable of resolving issues.
- Monitor and adjust retry budgets based on service performance.
How It Works
The error ownership mechanism distinguishes between errors originating from a service and those propagated from downstream services. By using a Service Dependency Analysis Solution, it claims or unclaims errors based on their source, allowing retries only for services that can address the underlying issue.
This prevents unnecessary load on struggling services and limits the spread of retry storms.