Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·10d ago·5 min read
Architectural Sampling: Test-Time Scaling via Computational Diversity in Frozen Vision-Language Models
TL;DR
Architectural sampling introduces a novel method for generating diverse responses from frozen vision-language models by reusing decoder blocks, improving candidate accuracy significantly. This approach enhances performance by 6.58 percentage points over traditional temperature sampling without altering model weights.
✦ Why It Matters
Implement architectural sampling to improve response diversity and accuracy in your vision-language models today.
Key Takeaways
- Explore architectural sampling to improve candidate generation in your models.
- Reuse early decoder layers for maximum performance gains.
- Utilize diverse forward computations to reduce lexical overlap.
- Apply findings to enhance label-free reinforcement learning strategies.
How It Works
Architectural sampling operates by reusing selected blocks of decoder layers in a vision-language model, allowing for distinct forward computations. This method introduces variability in the computation path by changing the block location and repetition count, which leads to diverse candidate outputs without altering the model's weights or adding new parameters.