EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Hugging Face Blog·3d ago·5 min read
Impactful scheduling for GPU clusters
TL;DR
Ai2's AI Infrastructure team revamped their GPU scheduling by implementing a budget-based system, significantly improving resource allocation and reducing wait times for workloads.
✦ Why It Matters
Consider implementing a budget-based resource allocation system to improve GPU scheduling in your projects.
Key Takeaways
- Implement a budget-based allocation for GPU resources.
- Use hierarchical fair-share scheduling to manage workloads.
- Track occupancy to ensure efficient resource use.
- Encourage transparent discussions about resource needs.
How It Works
The new scheduling system allocates GPU time based on project budgets, allowing managers to prioritize workloads strategically. Workloads are charged against their budgets, incentivizing teams to use resources efficiently and reducing the tendency to 'squat' on idle GPUs.
The scheduler also incorporates a time-slicing contract, allowing workloads to be preempted and requeued, which helps maintain high occupancy and ensures that resources are available for urgent tasks.