NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·1h ago
AI Research
arXiv CS.AI·Jun 11·4 min read
On the Geometry of On-Policy Distillation
TL;DR
Researchers identified a gap in LLM evaluation benchmarks. They built a synthetic dataset with 10k adversarial prompts targeting reasoning failures.
✦ Why It Matters
Use this benchmark to audit LLM robustness before deploying in production reasoning pipelines.
Key Takeaways
- Explore on-policy distillation for improved model reasoning.
- Utilize subspace locking to enhance training efficiency.
- Experiment with update token sparsity in OPD.
- Avoid mixing OPD with RLVR to maintain update dynamics.
How It Works
OPD updates are characterized by their unique trajectory in parameter space, which allows them to avoid principal directions and affect fewer weights. This results in a more efficient training process, as OPD updates quickly converge into a low-dimensional channel, known as subspace locking.
This mechanism preserves performance while reducing the complexity of the model's updates.