NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·3h ago
AI Research
arXiv CS.AI·May 25·8 min read
Score-Based One-step MeanFlow Policy Optimization
TL;DR
Prior flow-matching and diffusion policies in reinforcement learning require many sequential denoising steps at inference, creating computational bottlenecks in online RL where agents learn while interacting. Score-Based One-step MeanFlow Policy Optimization (SOM) constructs target velocity fields directly from Q-function estimates, enabling single-step policy generation.
✦ Why It Matters
Engineers can deploy faster RL policies in real-time control tasks by using single-step generation instead of iterative denoising.
Key Takeaways
- Explore SOM for efficient online reinforcement learning applications.
- Utilize Q-function for constructing target velocity fields.
- Reduce computational overhead in training and inference.
- Achieve state-of-the-art performance in locomotion tasks.
How It Works
SOM constructs the target velocity field directly from the Q-function, which estimates the expected future rewards for actions. By using score estimation and a probability flow ODE, SOM effectively directs the learning process towards high-value actions, allowing for a single-step generation that bypasses the need for multiple denoising steps.