Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·Aug 24·5 min read
Structured but Fragile: On the Limits of LLMs in Cybersecurity Decision-Making
TL;DR
Large language models (LLMs) can perform structured reasoning in cybersecurity decision-making but their effectiveness is fragile and sensitive to input framing. When given clear attack-graph structures, LLMs generate coherent strategies, yet small changes in prompts can lead to significant variations in outcomes.
✦ Why It Matters
Evaluate LLM outputs carefully, especially when framing prompts in cybersecurity decision-making tasks.
Key Takeaways
- Test LLMs with clear attack-graph structures for better outcomes.
- Be cautious of prompt sensitivity when using LLMs for decision-making.
- Consider using specialized solvers for optimization tasks over LLMs.
- Regularly assess LLM strategies against established benchmarks.
How It Works
LLMs analyze structured attack graphs to select security controls, aiming to minimize the success of potential attackers. By comparing their strategies to a game-theoretic optimization baseline, researchers assess the coherence and effectiveness of LLM-generated decisions.
⚠ The Catch
LLM performance is fragile; small changes in input prompts can lead to drastically different evaluations of strategies, undermining reliability in critical decision-making scenarios.