Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·Jun 9·5 min read
Beyond Pass/Fail: Using Process Mining to Understand How LLMs Resist (and Fail) Red Team Attacks
TL;DR
Standard evaluations of AI models in red teaming focus solely on whether attacks succeed or fail, ignoring the detailed process of resistance. This study applies process mining, a technique for analyzing sequences of events, to better understand how large language models (LLMs) respond to adversarial prompts.
✦ Why It Matters
Engineers can use process mining to enhance the robustness of AI models against adversarial attacks.
Key Takeaways
- Implement process mining to analyze model responses.
- Explore sequential patterns in adversarial interactions.
- Utilize insights to strengthen model defenses.
- Test various prompt strategies for comprehensive evaluation.
Full Summary
Red teaming in AI typically measures success through a binary attack success rate (ASR), which overlooks the complexity of how models resist or succumb to adversarial prompts. To address this, process mining is employed, allowing for a detailed analysis of the sequential interactions between prompts and model responses.
In a controlled experiment, 60 prompts from the HarmBench dataset were tested against two large language models: GPT-OSS 120B and Llama 3.3 70B, using 10 different prompt mutation strategies. The findings indicate that models exhibit varied resistance patterns, providing a richer understanding of their vulnerabilities and strengths.
This approach not only enhances the evaluation of LLMs but also informs the development of more robust models. The implications suggest that engineers can leverage these insights to improve model defenses against adversarial attacks.