Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·May 25·8 min read
Transcoders Trace Visual Grounding and Hallucinations in Vision-Language Models
TL;DR
Vision-language models (systems that process both images and text) produce outputs whose connection to visual inputs remains opaque. Researchers applied Transcoders—sparse neural network approximations that isolate individual computational functions—to decompose how Gemma 3-4B-IT converts image patches into generated tokens.
✦ Why It Matters
Engineers can now trace which image regions cause specific model outputs and detect hallucinations mechanistically, improving VLM reliability and debuggability.
Key Takeaways
- Transcoders outperform SAEs at identifying which image regions drive specific generated tokens.
- Function-centric decomposition reveals causal computational pathways in vision-language models.
- Graph-based mechanistic features from transcoder traces predict hallucinations at AUC 0.68.
- Counterfactual analysis confirms recovered pathways are specific to vision-language tasks, not general computation.
Full Summary
Vision-language models excel at multimodal reasoning but lack interpretability—it is unclear how images influence text generation. Prior work used Sparse Autoencoders (SAEs), which analyze static intermediate representations but miss the dynamic computational steps driving cross-modal interaction.
This work adopts Transcoders, sparse approximations of MLP (multilayer perceptron) sublayers that act as causal proxies for layer-wise computation. Applied to Gemma 3-4B-IT, the framework traces interpretable pathways linking image patches to token generation directions.
Transcoder attributions produced stronger, more stable effects on visually grounded tokens under patch ablation than SAE attributions and aligned better with semantically relevant image regions. A counterfactual analysis confirmed pathway specificity to vision-language tasks.
Graph-based mechanistic features extracted from transcoder circuit traces predicted hallucinations at AUC 0.68, demonstrating that function-centric decomposition yields both interpretable and predictive accounts of multimodal computation.