Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·Sep 2·5 min read
ReNFT: Repairing Mode Collapse in Reward Post-Training via Internal Probability-Mass Recalibration
TL;DR
ReNFT is a novel approach that addresses mode collapse in reward post-training of diffusion generators by recalibrating internal probability mass. It retains high reward levels while significantly improving diversity in generated outputs.
✦ Why It Matters
Implement ReNFT to enhance the diversity of your diffusion models without sacrificing reward performance.
Key Takeaways
- Use ReNFT to repair collapsed diffusion generators.
- Prioritize 'anti-hub' prompts for effective recalibration.
- Generate counterfactual proposals to explore diverse outputs.
- Retain high reward levels while improving output diversity.
How It Works
ReNFT recalibrates internal probability mass by using unconditional probes to identify prompts that expose biases. It generates matched counterfactual proposals from the same prompt and initial noise, allowing the model to explore suppressed alternatives and adjust its output tendencies.
This internal adjustment helps recover diversity without losing the benefits of previous training.