BerriAI / litellm
github.com·3d ago
Developer Tools
Databricks Blog·Jul 28·5 min read
How Databricks manages its own coding agent spend with Unity AI Gateway Budgets
TL;DR
Databricks has implemented a centralized governance model for AI spend by routing all coding agents through Unity AI Gateway, allowing for budget enforcement and visibility. This approach balances innovation with cost control, enabling self-service budget increases while preventing excessive spending.
✦ Why It Matters
Implement centralized budget controls for AI tools to prevent overspending and maintain productivity.
Key Takeaways
- Adopt a unified budget management system for coding agents.
- Implement daily and monthly spending limits to control costs.
- Allow engineers to self-raise daily limits to reduce interruptions.
- Encourage AI adoption by removing fear of unbounded costs.
How It Works
Databricks routes all coding agent traffic through Unity AI Gateway, allowing centralized budget enforcement. Engineers have a daily limit to catch runaway spending, which can be self-raised with a simple acknowledgment.
A higher monthly limit governs larger project expenses, requiring manager approval for increases. This dual-budget system ensures that normal usage is uninterrupted while still providing safeguards against excessive costs.