Reimagining service delivery in the agentic era with Google Public Sector
cloud.google.com·21h ago
AI Research
arXiv CS.AI·Jun 10·5 min read
Improving Adversarial Transferability on Vision-Language Pre-training Models via Surrogate-Specific Bias Correction
TL;DR
Vision-Language Pre-training (VLP) models are vulnerable to adversarial attacks, which can be less effective across different models due to a reliance on surrogate models. The DeBias-Attack method was developed to correct this issue by optimizing adversarial perturbations while addressing surrogate-specific bias.
✦ Why It Matters
Engineers can leverage DeBias-Attack to enhance the robustness of VLP models against adversarial attacks.
Key Takeaways
- Utilize DeBias-Attack to improve adversarial robustness in VLP models.
- Implement two perturbation branches for effective adversarial optimization.
- Correct surrogate-specific bias to enhance transferability of attacks.
- Test the method across various VLP models and tasks for best results.
How It Works
DeBias-Attack operates by maintaining two branches for perturbation: the main branch optimizes on the original image, while the reference branch uses a weak-semantic image. This weak image, constructed from the dataset mean with added noise, helps estimate the surrogate-specific bias.
By removing the aligned projection of the main gradient on the reference gradient, the method corrects the bias, leading to improved adversarial transferability.