NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·3h ago
AI Research
arXiv CS.AI·Jun 10·5 min read
SD-GRPO: Verifiable Segment Decomposition for Long-Form Vision-Language Generation
TL;DR
Long-form vision-language (VL) generation tasks often suffer from inadequate credit assignment due to their complex outputs. To address this, Segment-Decomposed GRPO (SD-GRPO) was developed, which normalizes rewards for each segment of the output instead of using a single scalar value.
✦ Why It Matters
Engineers can enhance long-form vision-language generation models by implementing SD-GRPO for better credit assignment.
Key Takeaways
- Implement SD-GRPO to improve long-form VL generation tasks.
- Normalize rewards for each segment to enhance performance.
- Combine holistic and per-segment rewards for semantically entangled outputs.
- Integrate SD-GRPO into existing GRPO frameworks with minimal overhead.
Full Summary
Long-form vision-language generation involves creating detailed textual descriptions based on complex images, which can lead to challenges in accurately assigning credit for different segments of the output. Traditional methods like Group Relative Policy Optimization (GRPO) use a single scalar advantage for credit assignment, which can underfit these tasks.
The new method, Segment-Decomposed GRPO (SD-GRPO), normalizes rewards for each segment, allowing for a more nuanced understanding of performance. Evaluations across various tasks, including controlled multi-panel dense-captioning and real-world scientific figure captioning, demonstrated that SD-GRPO consistently outperformed GRPO, particularly as the number of segments increased.
In cases where segments were semantically entangled, combining holistic and per-segment rewards further enhanced performance. This approach can be integrated into existing GRPO frameworks with minimal effort, making it a versatile tool for improving long-form VL generation.