Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·May 25·8 min read
AI Evaluation Should Require Standardized Item-Level Data Releases
TL;DR
AI benchmarks currently report only aggregate scores, hiding flawed test items and misaligned evaluation constructs—problems that inflate capability claims. Researchers built OpenEval, an archive of 10 million model responses across 155,000 benchmark items with standardized metadata, enabling detailed analysis of evaluation quality.
✦ Why It Matters
Engineers can demand and use item-level benchmark data to validate AI system claims before deployment, preventing costly decisions based on inflated capability metrics.
Key Takeaways
- Require item-level response data releases alongside benchmark scores to enable validity assessment.
- Use OpenEval schema or similar standardized formats when publishing evaluation results for reproducibility.
- Analyze individual item performance to identify low-quality test items before trusting benchmark conclusions.
- Audit benchmarks for construct misalignment—verify they measure what they claim to measure.
Full Summary
Current AI evaluation relies on aggregate model scores (overall accuracy or performance metrics), which obscure critical problems: poorly selected test items, misalignment between what benchmarks claim to measure and what they actually measure, and weak generalization to real-world performance. This opacity leads to inflated capability claims and misdirected research priorities.
The authors constructed OpenEval, a standardized archive containing 10 million individual model responses across 155,000 test items from widely-used benchmarks, organized under a unified schema for community development. Using item-level analysis, they identified low-quality items that don't discriminate between models, documented construct misalignment (where benchmarks measure something different than intended), and recovered validity evidence about benchmark internal structure.
The work addresses practical concerns: data contamination risks and author burden are tractable compared to the cost of decisions based on untrustworthy evaluation claims. Item-level data releases enable transparency, replicability, and auditability—essential for responsible AI deployment.