Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·May 25·4 min read
SUDP: Secret-Use Delegation Protocol for Agentic Systems
TL;DR
Researchers identified a gap in LLM evaluation benchmarks. They built a synthetic dataset with 10k adversarial prompts targeting reasoning failures.
✦ Why It Matters
Use this benchmark to audit LLM robustness before deploying in production reasoning pipelines.
Key Takeaways
- Adopt SUDP to prevent unauthorized access to user secrets.
- Utilize unique grants for each operation to enhance security.
- Integrate SUDP with hardware-rooted environments for optimal protection.
- Ensure compliance with the seven security properties outlined in SUDP.
How It Works
SUDP allows a requester to propose an operation, which the user then authorizes using a fresh authenticator-backed grant. This grant is redeemed by a custodian to perform the operation, ensuring that the requester never gains reusable authority over the user's secrets.