Agentic data science with the Data Agent Kit
cloud.google.com·8h ago
AI Research
arXiv CS.AI·Aug 4·5 min read
Grounding Agentic VLMs with Dedicated Segmentation for Fine-Grained Vehicle Damage Assessment
TL;DR
TinyDamage is a novel hybrid architecture that enhances vehicle damage assessment by combining a vision-language model (VLM) with a dedicated segmentation model, significantly reducing report hallucination rates.
✦ Why It Matters
Implement TinyDamage to improve the accuracy of automated vehicle damage assessments in your applications today.
Key Takeaways
- Integrate segmentation models for improved spatial grounding in VLMs.
- Use supervised contrastive loss for better tiny-object detection.
- Adopt TinyDamage to reduce report inaccuracies in vehicle assessments.
- Evaluate tiny-object grounding with the new DET_l metric.
How It Works
TinyDamage combines a vision-language model (VLM) for semantic reasoning with a dedicated segmentation model that focuses on accurately identifying fine-grained vehicle damages. The segmentation model uses a supervised contrastive loss function, which enhances the separation between damage and background, leading to better detection of small defects.
The integration into a LangGraph agent pipeline ensures that every output from the VLM is grounded in the segmentation results, improving overall accuracy.