BerriAI / litellm
github.com·3d ago
Developer Tools
GitHub Trending·Sep 4·5 min read
radixark / miles
TL;DR
Miles v0.1 has been released, introducing production-level post-training capabilities for reinforcement learning (RL) models. Key updates include support for 8-bit and 4-bit RL and on-policy distillation.
✦ Why It Matters
Explore Miles v0.1 to enhance your RL model's efficiency and performance today.
Key Takeaways
- Utilize Miles for efficient post-training of large-scale models.
- Implement low-precision training to improve stability and performance.
- Leverage fault tolerance features to minimize downtime during training.
- Explore day-0 support for the latest models to stay ahead in research.
Full Summary
Miles is an enterprise-ready reinforcement learning (RL) framework that focuses on post-training for large-scale models, utilizing SGLang for high-throughput rollouts and Megatron-LM for scalable training. Key features include fully asynchronous RL, fast weight updates, and support for low-precision training formats like MXFP8 and NVFP4, which help maintain stability during training.
The framework also introduces innovative techniques such as token-in-token-out (TITO) and Rollout Routing Replay (R3) to optimize performance and reduce computational costs. With day-0 support for cutting-edge models like DeepSeek-V4 and Kimi-K3, Miles is positioned to handle the demands of modern AI research and applications.
Its extensive hardware compatibility ensures it can be deployed across various platforms, making it a versatile tool for researchers and developers alike.