Announcing Native BM25 Ranking in AlloyDB and Cloud SQL
cloud.google.com·1d ago
AI Research
arXiv CS.AI·Jun 12·5 min read
SMSR: Certified Defence Against Runtime Memory Poisoning in Persistent LLM Agent Systems
TL;DR
Persistent memory in retrieval-augmented generation (RAG) agents is vulnerable to Multi-Session Memory Poisoning (MSMP), where adversaries can inject harmful memories. To combat this, Signed Memory with Smoothed Retrieval (SMSR) was developed, which includes HMAC-SHA256 provenance checks and randomized memory ablation.
✦ Why It Matters
Engineers can implement SMSR to enhance the security of persistent memory in AI systems against memory poisoning attacks.
Key Takeaways
- Adopt SMSR to enhance security against memory poisoning in LLMs.
- Utilize HMAC-SHA256 for memory provenance to block unauthorized injections.
- Implement randomized memory ablation to mitigate adversarial influence.
- Conduct regular security assessments to evaluate defense effectiveness.
How It Works
SMSR operates through two main components: first, it uses HMAC-SHA256 to ensure that only authenticated memory can be written, effectively preventing unsigned memory injections. Second, during query time, it employs randomized memory ablation combined with majority voting to limit the impact of any adversarially injected memories, ensuring that even if an adversary manages to inject a memory, its influence is minimized through a voting mechanism that favors consistent, legitimate responses.