BerriAI / litellm
github.com·3d ago
Developer Tools
Databricks Blog·Sep 9·5 min read
Evaluation-First AI Agents: How Zepto Scales Customer Support on Databricks and MLflow
TL;DR
Zepto developed a multi-agent AI system for customer support that processes over 100,000 tickets daily, focusing on evaluation to ensure reliability as demand scales. This approach minimized errors and improved customer satisfaction during peak times.
✦ Why It Matters
Implement an evaluation-first strategy in your AI systems to enhance reliability and customer satisfaction.
Key Takeaways
- Prioritize evaluation in AI agent development to ensure reliability.
- Use Databricks and MLflow for effective model testing and deployment.
- Monitor ticket volume spikes to adapt support strategies.
- Focus on diverse customer needs to minimize failure modes.
Full Summary
Zepto, a rapidly growing quick-commerce platform in India, faced the challenge of maintaining reliable customer support as ticket volumes surged. To address this, they implemented a multi-agent AI system that processes over 100,000 support tickets each day.
Instead of simply increasing the number of agents, Zepto prioritized an evaluation-first methodology using Databricks and MLflow, which allowed for rigorous testing and operational oversight. This approach proved crucial during high-demand periods, such as weather events and holidays, where even a 1% error rate could lead to significant revenue loss.
The partnership with Databricks enabled Zepto to adapt to changing customer behaviors and diverse support requests, ultimately enhancing the reliability of their AI agents. The results demonstrated that a focus on evaluation could lead to better performance and customer satisfaction.