NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·Jun 10·5 min read
Dropout-GRPO: Variational Stochasticity for Continuous Latent Reasoning
TL;DR
Continuous latent reasoning models like Coconut struggle with generating diverse outputs due to deterministic rollouts, which hinder performance. To overcome this, a new method called Dropout-GRPO introduces structured dropout to create variability in trajectories.
✦ Why It Matters
Engineers can leverage Dropout-GRPO to enhance the performance of continuous latent reasoning models in practical applications.
Key Takeaways
- Explore structured dropout to enhance model variability.
- Apply Dropout-GRPO to continuous latent reasoning tasks.
- Evaluate performance improvements on benchmarks like GSM8K.
- Consider Bayesian model averaging for better optimization.
How It Works
Dropout-GRPO introduces a single Bernoulli mask that remains constant across all steps of a rollout. This mask generates variability in the trajectories produced by the model, allowing GRPO to function effectively despite the deterministic nature of continuous latent reasoning models.
By treating each rollout as a sample from a variational distribution, the method optimizes the expected reward, leading to improved performance.