Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·Jun 15·5 min read
Mirage Probes: How Vision Models Fake Visual Understanding
TL;DR
Vision-language models (VLMs) often provide confident answers to image-based questions even without images, misleadingly inflating performance metrics. The authors developed Mirage Probes, a contrastive probing framework, to analyze this behavior and identify two distinct failure modes: textual biases and spurious images.
✦ Why It Matters
Engineers must address both textual biases and spurious images to improve VLM reliability and visual grounding.
Key Takeaways
- Implement Mirage Probes to analyze model behavior.
- Distinguish between textual biases and spurious images.
- Use Prior Harnessing Index to evaluate model reliance on text.
- Focus on representational changes for better visual grounding.
Full Summary
Vision-language models (VLMs) are designed to interpret and respond to questions about images, but they can generate confident answers even when no image is present, a phenomenon termed 'mirage behavior.' To investigate this, the authors introduced Mirage Probes, a contrastive probing framework that pairs paraphrased questions with both mirage and non-mirage labels for the same image.
Their analysis of two open-source VLMs showed that mirage behavior can be decoded from various internal activations, indicating two failure modes: textual biases, where answers derive from language alone, and spurious images, where the model fabricates visual content. They developed a Prior Harnessing Index (PHI) to measure how much a model relies on text, revealing that cleaning text distributions can mitigate textual biases but not spurious images.
This distinction suggests that improving visual grounding in VLMs requires deeper changes to the model's visual representations. The implications are significant for engineers aiming to enhance model reliability and accuracy.