NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·3h ago
AI Research
arXiv CS.AI·Jun 12·5 min read
Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoning
TL;DR
Recent vision-language models struggle with spatial reasoning tasks that require active evidence gathering. PERception-Interaction-reason Agent (PERIA) was developed to enhance spatial reasoning through tool-augmented interactions.
✦ Why It Matters
Engineers can leverage PERIA's tool-augmented approach to improve spatial reasoning in their AI applications.
Key Takeaways
- Explore tool-augmented models for enhanced spatial reasoning.
- Implement vision perception tools for evidence gathering.
- Utilize OR-GIGPO for effective multi-tool training.
- Benchmark against state-of-the-art models for performance evaluation.
Full Summary
Vision-language models (VLMs) have shown strong performance in multimodal understanding but fall short in spatial reasoning tasks that necessitate active evidence acquisition and multi-step interactions. To address this, PERception-Interaction-reason Agent (PERIA) was created, which utilizes two types of lightweight tools: vision perception tools for revealing textual, symbolic, and spatial evidence, and vision interaction tools for manipulating visual contexts and verifying spatial relationships.
The training methodology combines supervised tool-use trajectory synthesis, composite rewards, and Observation-Relaxed Group-in-Group Policy Optimization (OR-GIGPO) to facilitate effective multi-tool behavior. In experiments across 13 benchmarks from 8 datasets, PERIA-8B improved performance by 10.0% on in-distribution benchmarks and 4.4% on out-of-distribution benchmarks compared to the Qwen3-8B backbone.
It also surpassed previous state-of-the-art models of similar size by 7.0%-14.8% and achieved results comparable to much larger models like Qwen3-VL-235B-A22B-Thinking and GPT-5. These findings suggest that tool-augmented approaches can significantly enhance spatial reasoning capabilities in visual agents.