Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·May 25·6 min read
Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?
TL;DR
Vision-language models (VLMs)—AI systems combining image and text understanding—score well on standard benchmarks, but it's unclear whether these scores reflect genuine visual understanding. Researchers systematically degraded images by removing tokens (data chunks) and tested multiple VLM architectures across various evaluation methods including occlusion and layer-wise analysis.
✦ Why It Matters
Engineers must redesign VLM evaluation methods to ensure models actually use visual information rather than relying on shortcuts.
Key Takeaways
- Current VLM benchmarks don't measure genuine visual understanding; high accuracy can mask token-independent reasoning.
- Test VLMs by systematically removing or occluding image regions to verify visual sensitivity beyond standard metrics.
- Examine internal model representations across layers to detect when visual information becomes redundant or ignored.
- Develop benchmarks requiring fine-grained visual discrimination rather than coarse-grained pattern matching for robust evaluation.
Full Summary
Vision-language models (VLMs) are AI systems trained to understand both images and text together. Standard benchmarks measure their accuracy on tasks like answering questions about images, but researchers questioned whether high scores actually indicate the models truly rely on visual evidence.
The team discovered that removing a large fraction of image tokens—discrete units of visual information—caused only minimal performance degradation on a hallucination benchmark (a test measuring whether models make false claims). They conducted systematic experiments across multiple open-source VLMs using global image degradation, localized occlusion (blocking parts of images), question reformulation, and layer-wise geometric analysis of vision tokens.
Even when final predictions remained correct, internal model representations showed weakened support for answers. Analysis revealed that visual tokens become increasingly similar in deeper network layers, explaining why models can maintain accuracy without fine-grained visual details.
These findings indicate current benchmarks fail to reliably assess whether VLMs genuinely ground their reasoning in visual evidence.