Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·Aug 10·5 min read
LoRAScan: Detecting Backdoor Prompts in Low-Rank Adapters for Large Language Models via Down-Projection Activation Spikes
TL;DR
LoRAScan is a novel defense mechanism that detects backdoor prompts in low-rank adapters for large language models by identifying activation spikes. It achieves a 98.49% rejection rate of malicious inputs while maintaining a low error rate on clean inputs.
✦ Why It Matters
Implement LoRAScan to enhance the security of your language model deployments against backdoor attacks.
Key Takeaways
- Utilize LoRAScan to detect backdoor prompts in language models.
- Monitor low-variance insertion sites for security.
- Achieve high rejection rates of malicious inputs with minimal errors.
- Enhance model deployment security against supply-chain threats.
How It Works
LoRAScan identifies a small subset of LoRA insertion sites that remain stable across clean inputs but show concentrated spikes in activation when a trigger is present. By monitoring these sites during inference, it can effectively detect and reject malicious inputs without altering the adapter's parameters.