Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·25d ago·5 min read
Adversarial Decoys: Misdirecting Attention-Based Defenses in ViT
TL;DR
Attention-based defenses in Vision Transformers (ViT) can be misled by adversarial decoys, which are strategically crafted inputs. These decoys exploit the model's attention mechanisms to divert focus away from actual adversarial examples.
✦ Why It Matters
Engineers should consider integrating adversarial decoy detection mechanisms into their ViT models to improve security.
Key Takeaways
- Explore adversarial decoys to enhance attack effectiveness in ViTs.
- Decouple misclassification from attention manipulation in adversarial strategies.
- Test the impact of attention scores on your model's vulnerability.
- Implement layer-wise objectives to optimize attention on target tokens.
How It Works
Adversarial decoys are optimized independently from the main adversarial attack, allowing them to redirect attention towards specific target tokens. By increasing the attention scores of these tokens, the decoys effectively mislead defenses that rely on attention magnitude to identify adversarial regions.
This decoupling of objectives enables the original adversarial attack to remain potent while the decoys manipulate the attention ranking.