NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·Jun 10·5 min read
Improving Adversarial Transferability on Vision-Language Pre-training Models via Surrogate-Specific Bias Correction
TL;DR
Vision-Language Pre-training (VLP) models are vulnerable to adversarial attacks, which can be less effective across different models due to a reliance on surrogate models. The DeBias-Attack method was developed to correct this issue by optimizing adversarial perturbations while addressing surrogate-specific bias.
✦ Why It Matters
Engineers can leverage DeBias-Attack to enhance the robustness of VLP models against adversarial attacks.
Key Takeaways
- Implement DeBias-Attack to improve adversarial transferability.
- Utilize two perturbation branches for better optimization.
- Focus on correcting surrogate-specific bias in adversarial training.
- Test robustness across various VLP models and tasks.
Full Summary
Adversarial examples expose weaknesses in Vision-Language Pre-training (VLP) models, which combine visual and textual data. A major challenge is that existing adversarial attacks often depend too much on surrogate models, leading to poor performance when applied to different models.
To address this, DeBias-Attack was introduced, which employs two perturbation branches: one optimizes perturbations on the original image, while the other uses a weak-semantic image to estimate surrogate-specific bias. By removing the aligned projection of the main gradient on the reference gradient, DeBias-Attack enhances the adversarial image's effectiveness.
Experimental results showed that this method outperformed previous techniques, achieving strong results across multiple VLP models and tasks, including both open-source and closed-source multimodal large language models. These findings suggest that correcting for surrogate-specific bias can lead to more robust adversarial attacks in VLP contexts.