Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·May 25·8 min read
MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detection
TL;DR
LLM agents with persistent memory face a security risk: attackers can inject malicious records through normal interactions, which later influence agent behavior. MemAudit, a post-hoc auditing framework, identifies poisoned memories using causal influence scoring and structural anomaly detection.
✦ Why It Matters
Engineers can now detect and remove poisoned memories after attacks occur, closing a critical security gap in deployed LLM agents.
Key Takeaways
- Adopt MemAudit for post-hoc memory auditing in AI agents.
- Utilize causal attribution to identify harmful memory contributions.
- Implement structural anomaly detection to enhance memory integrity.
- Reduce vulnerability to adversarial memory attacks significantly.
How It Works
MemAudit combines two innovative techniques: a counterfactual memory influence score, which assesses how much each memory entry contributes to harmful outputs, and a memory consistency graph that identifies memories that deviate from expected patterns. This dual approach allows for a comprehensive analysis of the memory's integrity and the identification of malicious entries.