NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·Jun 12·5 min read
Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents
TL;DR
Tool-using agents based on large language models (LLMs) face significant safety risks, especially during multi-turn interactions. To address this, researchers developed MT-AgentRisk, a benchmark for evaluating multi-turn safety, and ToolShield, a self-exploration defense mechanism.
✦ Why It Matters
Engineers can implement ToolShield to enhance the safety of multi-turn interactions in tool-using agents.
Key Takeaways
- Evaluate agent safety using the MT-AgentRisk benchmark.
- Implement ToolShield for autonomous safety testing.
- Monitor ASR to assess safety in multi-turn scenarios.
- Explore self-exploration techniques for tool-using agents.
Full Summary
As large language model (LLM)-based agents become more capable, their safety in practical applications remains a concern, particularly in multi-turn interactions where they use various tools. Existing benchmarks do not adequately address the unique risks that arise in these complex scenarios.
To fill this gap, researchers created MT-AgentRisk, the first benchmark designed to evaluate the safety of tool-using agents in multi-turn settings. They also introduced ToolShield, a training-free, tool-agnostic defense mechanism that allows agents to autonomously generate and test scenarios with new tools to learn from their experiences.
Experimental results indicated a significant safety degradation, with the Attack Success Rate (ASR) increasing by 16% in multi-turn interactions. However, ToolShield effectively mitigated this issue, reducing ASR by an average of 30%.
These findings highlight the importance of developing robust safety measures for agents operating in dynamic environments.