Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·Sep 9·5 min read
Characterizing Contention-Induced Reliability Collapse in KV-Cache Timing Side Channels for Multi-Tenant LLM Serving
TL;DR
Research reveals that contention in shared key-value (KV) caches during multi-tenant large language model (LLM) serving significantly reduces the reliability of timing side channel attacks. Key metrics show a drop in attack effectiveness as worker contention increases, highlighting vulnerabilities in LLM systems under load.
✦ Why It Matters
Evaluate your LLM serving architecture for potential timing side channel vulnerabilities under multi-tenant conditions.
Key Takeaways
- Assess the impact of worker contention on LLM serving reliability.
- Implement measures to mitigate timing side channel vulnerabilities.
- Monitor concurrency-depth variance to understand attack effectiveness.
- Consider the implications of shared caching in multi-tenant environments.
How It Works
The study examines how shared KV caches can create timing side channels that leak information about cached data. By measuring the effect size of attacks under varying levels of contention, the researchers identify a non-monotonic relationship between worker overlap and attack reliability.