Reimagining service delivery in the agentic era with Google Public Sector
cloud.google.com·19h ago
AI Research
arXiv CS.AI·Jun 9·5 min read
FiberTune: Preserving Action-Fiber Visual Residuals in Vision-Language-Action Fine-Tuning
TL;DR
Action-supervised fine-tuning of vision-language-action (VLA) policies often leads to a loss of visual structure across similar actions. FiberTune is a new training objective designed to maintain visual residuals during this fine-tuning process without increasing inference time.
✦ Why It Matters
Engineers can improve VLA policy robustness by implementing FiberTune to maintain visual consistency during training.
Key Takeaways
- Implement FiberTune to preserve visual residuals during fine-tuning.
- Use an online action probe to filter action-predictive features.
- Align filtered visual representations with a frozen teacher model.
- Expect significant performance improvements in VLA tasks.
How It Works
FiberTune operates by using an online action probe to identify and filter out features that predict actions from visual representations. This filtering process helps maintain the integrity of visual information that is crucial for understanding actions, thereby preventing the collapse of visual structure across similar action states.
The filtered representations are then aligned with a frozen visual teacher model, which serves as a reference for maintaining high-quality visual features during training.