EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Hugging Face Blog·Sep 10·5 min read
Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL
TL;DR
Async GRPO with LoRA enables efficient training of lightweight adapters in Hugging Face Jobs, significantly reducing training time from over three hours to under an hour for 500 steps.
✦ Why It Matters
Implement Async GRPO with LoRA to drastically reduce your model training time today.
Key Takeaways
- Utilize AsyncGRPOTrainer for efficient LoRA adapter training.
- Sync only small adapter files to reduce data transfer overhead.
- Implement a proxy server to optimize rollout requests.
- Leverage Storage Buckets for shared filesystem access across jobs.
How It Works
The AsyncGRPOTrainer trains LoRA adapters, which are small enough to be stored in a Storage Bucket, allowing for quick syncing to vLLM. A proxy server routes requests to the appropriate vLLM replica, ensuring efficient use of cached data and minimizing redundant computations.