Gemini 4 Carbon: Our next frontier model
deepmind.google·5d ago
AI Engineering
Hacker News·Jul 20·5 min read
LoRA Speedrun – a public wall-clock leaderboard for fine-tuning techniques
TL;DR
A public leaderboard has been established for fine-tuning the Qwen2.5-1.5B model using LoRA (Low-Rank Adaptation) techniques. Participants can compete on a Modal L40S hardware setup, with results verified through multiple runs.
✦ Why It Matters
Engineers can participate in the LoRA Speedrun to benchmark their fine-tuning techniques against peers and improve their models.
Key Takeaways
- Experiment with LoRA to fine-tune models efficiently.
- Utilize Modal's free credits for training and verification.
- Aim to beat the current record of 1m 44s for recognition.
- Document your methods thoroughly for leaderboard submissions.
How It Works
Participants fine-tune the Qwen2.5-1.5B model using LoRA, which involves training a small adapter on top of a frozen model. The leaderboard tracks wall-clock time, ensuring that all techniques are evaluated under the same conditions.
Each submission undergoes rigorous verification, requiring three successful runs to establish a record.