Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·Jul 29·5 min read
RefBench-PRO: Perceptual and Reasoning Oriented Benchmark for Referring Expression Comprehension
TL;DR
RefBench-PRO is a new benchmark for Referring Expression Comprehension (REC) that evaluates both perceptual and reasoning capabilities of Multi-modal Large Language Models (MLLMs). It introduces six challenging tasks and a novel RL-based learning scheme to enhance localization accuracy.
✦ Why It Matters
Implement RefBench-PRO to rigorously evaluate your MLLM's performance in referring expression tasks.
Key Takeaways
- Utilize RefBench-PRO for comprehensive REC evaluation.
- Incorporate RL techniques to enhance model localization accuracy.
- Explore the six sub-tasks for targeted model training.
- Leverage automated data generation for diverse training sets.
How It Works
RefBench-PRO decomposes referring expressions into perception and reasoning dimensions, creating six progressively challenging tasks. The automated data-generation pipeline ensures a wide variety of expressions, while the Ref-R1 learning scheme employs Dynamic IoU to refine localization under complex reasoning conditions.