Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·May 26·5 min read
Who judges the judges? Governance from metrics: a runtime framework for continuous LLM compliance monitoring
TL;DR
Current AI compliance methods treat adherence as a one-time check rather than an ongoing process. The authors developed govllm, an open-source framework that continuously monitors compliance using runtime metrics and a panel of specialized evaluators.
✦ Why It Matters
Engineers can leverage govllm to implement continuous compliance monitoring in AI systems, enhancing regulatory adherence.
Key Takeaways
- Adopt continuous compliance monitoring frameworks like govllm.
- Utilize runtime metrics for ongoing AI system evaluation.
- Incorporate specialized evaluators for regulatory compliance.
- Recognize inter-judge disagreement as a valuable signal.
Full Summary
AI compliance traditionally relies on static assessments, which do not align with the EU AI Act's requirement for continuous oversight. To address this, the authors introduced govllm, a framework that utilizes runtime observability to derive compliance metrics.
It employs a governance-driven routing architecture where model selection is based on compliance scores rather than just performance metrics like latency. A panel of regulatory judges evaluates models against criteria such as the EU AI Act and GDPR, with findings indicating that inter-judge disagreement can signal regulatory uncertainty.
The study validated this approach using a dataset of 49 annotated prompt/response pairs evaluated by four small language models, achieving agreement rates between 51.5% and 69.1%. Additionally, the research identified structural failure modes in regulatory judges and biases affecting agreement rates.
This framework is released as open-source software to facilitate reproducible research in AI governance.