Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·25d ago·5 min read
Trajectory Learnability for Offline On-Policy Distillation with Imperfect Teachers
TL;DR
A novel method for offline on-policy distillation leverages teacher-successful problems to enhance learning from imperfect teacher supervision, achieving up to 2.7 percentage points improvement over traditional methods. This approach maintains efficiency while reducing resource usage compared to online variants.
✦ Why It Matters
Implement this method to improve your model's performance with less computational overhead today.
Key Takeaways
- Utilize teacher-successful problems to enhance learning from failures.
- Analyze token likelihood changes to derive learnability signals.
- Reduce resource usage while maintaining model performance.
- Consider offline methods for efficiency in training.
How It Works
The method defines a learnability signal based on the likelihood of tokens in trajectories from teacher-failed problems, using successful teacher problems as a reference. This allows for trajectory-level weights to be computed without additional generation, maintaining the efficiency of offline distillation.