Trusted Data Apps for the Agentic Data Cloud, powered by Looker
cloud.google.com·7h ago
System Design
ByteByteGo·Aug 3·5 min read
LLM Security Basics: The Full Threat Model
TL;DR
In June 2025, researchers revealed that Microsoft 365 Copilot could unintentionally leak internal files via a crafted email, highlighting vulnerabilities in language models. This incident, dubbed EchoLeak, underscores the need for robust security measures in AI systems.
✦ Why It Matters
Review your AI systems for potential vulnerabilities similar to EchoLeak to enhance security.
Key Takeaways
- Implement strict input validation to prevent prompt injection.
- Monitor and sanitize model outputs before downstream use.
- Limit agent permissions to reduce exposure to attacks.
- Ensure provenance of models and tools to mitigate supply chain risks.
How It Works
Language models process all input as a single sequence of tokens, allowing any part of the input to influence the output. This lack of separation between commands and data enables prompt injection attacks, where malicious instructions can be embedded in seemingly benign content.