Agentic data science with the Data Agent Kit
cloud.google.com·8h ago
AI Research
arXiv CS.AI·Aug 20·5 min read
FraudBench: Stress-Testing Policy-Grounded Banking Agents Against Adaptive Fraud
TL;DR
FraudBench is a new benchmark designed to evaluate the safety of policy-grounded banking agents against adaptive fraud tactics. Initial tests show that these agents have a security success rate between 49% and 65%, highlighting vulnerabilities in handling fraud scenarios.
✦ Why It Matters
Evaluate your banking agents against adaptive fraud scenarios using FraudBench to enhance security measures today.
Key Takeaways
- Implement FraudBench to test your banking agents against adaptive fraud scenarios.
- Focus on improving agent responses to money-mule and first-party fraud vulnerabilities.
- Utilize the annotated scenarios for better training and evaluation of agents.
- Consider the history-dependent nature of safety in agent interactions.
How It Works
FraudBench operates by simulating interactions between a banking agent and a caller, where both can manipulate account states and access internal policies. The benchmark includes adversarial scenarios that challenge the agent's ability to maintain security while responding to potentially deceptive requests.