Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·May 28·8 min read
Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent Systems
TL;DR
Multi-agent systems using large language models (LLMs) risk exposing sensitive information when agents communicate with each other, creating a privacy vulnerability not yet thoroughly evaluated. Researchers developed evaluation methods and benchmarks to measure how readily LLM agents leak confidential data during multi-agent interactions.
✦ Why It Matters
Engineers deploying multi-agent LLM systems must implement explicit privacy controls before handling sensitive data in collaborative workflows.
Key Takeaways
- Audit multi-agent LLM workflows for unintended secret disclosure before production deployment.
- Add explicit privacy constraints and access controls to agent communication protocols.
- Test agents with confidentiality benchmarks to measure baseline privacy failure rates.
- Implement agent-level data compartmentalization to limit sensitive information exposure.
Full Summary
Large language models deployed as autonomous agents in multi-agent systems—where multiple AI agents collaborate to solve tasks—face an understudied privacy risk: unintended disclosure of sensitive information during inter-agent communication. Prior work focused on single-agent privacy or external data leakage, but not on information leakage between agents themselves.
Researchers created evaluation frameworks and benchmark datasets to systematically measure privacy failures in multi-agent LLM systems, testing how easily agents reveal confidential data when instructed to keep secrets. Experiments across multiple LLM architectures and agent configurations showed consistent and substantial privacy breaches—agents frequently disclosed sensitive information despite explicit confidentiality instructions.
Findings indicate that standard LLM training and prompting techniques provide insufficient safeguards for multi-agent collaboration. These results highlight a critical gap in deploying LLM agents in real-world scenarios involving proprietary or personal data.