Agentic data science with the Data Agent Kit
cloud.google.com·8h ago
AI Research
arXiv CS.AI·Aug 11·5 min read
PAST: Privileged Adaptation from Complete Student Trajectories for On-Policy Self-Distillation
TL;DR
PAST introduces a novel approach to on-policy self-distillation (OPSD) by utilizing complete student trajectories as privileged information, leading to a 5.6 percentage point improvement in performance on mathematical reasoning tasks compared to standard OPSD.
✦ Why It Matters
Implement PAST in your OPSD frameworks to enhance model performance on reasoning tasks.
Key Takeaways
- Incorporate complete student trajectories for better model training.
- Use student-proximity regularization to adapt teacher models.
- Evaluate performance improvements on reasoning tasks with PAST.
- Conduct factorial studies to assess the impact of trajectory access.
How It Works
PAST enhances OPSD by treating each completed student trajectory as privileged information. It maintains the student's next-token distribution on successful paths while adapting the teacher model based on failed trajectories.
This dual approach allows the teacher to learn from both successes and failures, improving its guidance for the student.