NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·Jun 12·5 min read
SMSR: Certified Defence Against Runtime Memory Poisoning in Persistent LLM Agent Systems
TL;DR
Persistent memory in retrieval-augmented generation (RAG) agents is vulnerable to Multi-Session Memory Poisoning (MSMP), where adversaries can inject harmful memories. To combat this, Signed Memory with Smoothed Retrieval (SMSR) was developed, which includes HMAC-SHA256 provenance checks and randomized memory ablation.
✦ Why It Matters
Engineers can implement SMSR to enhance the security of persistent memory in AI systems against memory poisoning attacks.
Key Takeaways
- Implement HMAC-SHA256 for memory provenance in AI systems.
- Use randomized memory ablation to mitigate memory poisoning risks.
- Test SMSR in enterprise scenarios to validate effectiveness.
- Monitor attack success rates to assess security improvements.
Full Summary
Retrieval-augmented generation (RAG) agents, which utilize persistent memory across user sessions, face a new security threat known as Multi-Session Memory Poisoning (MSMP). This vulnerability allows adversaries to inject malicious memories that can influence future responses without altering the underlying model.
To address this, Signed Memory with Smoothed Retrieval (SMSR) was created, incorporating two main components: HMAC-SHA256 for memory provenance at write time, and randomized memory ablation with majority voting at query time. Testing across 15 enterprise scenarios showed that SMSR effectively blocked unsigned memory injections, reducing attack success from 93-100% to 0%.
For authenticated adversaries, SMSR limited success to 8.0% with a single injection, and in end-to-end attacks, it decreased success from 65.3% to 5.3%. These results demonstrate SMSR's robust defense capabilities while maintaining high utility for clean queries, at 90% and 85% respectively.