Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·Jun 12·5 min read
Cross-Model Disagreement as a Label-Free Correctness Signal
TL;DR
Detecting errors in language models without relying on labeled data is challenging, especially when models are confidently wrong. Cross-Model Perplexity (CMP) and Cross-Model Entropy (CME) were developed to measure how surprised a second model is by the output of a first model, providing a label-free correctness signal.
✦ Why It Matters
Engineers can implement CMP and CME to improve error detection in language models without needing labeled data.
Key Takeaways
- Integrate CMP and CME for better error detection in models.
- Use cross-model disagreement to enhance deployment monitoring.
- Apply these techniques without additional training requirements.
- Leverage label-free correctness signals in production systems.
Full Summary
Language models often produce confident but incorrect outputs, making it difficult to ensure their reliability without ground truth labels. To address this, CMP and CME were introduced as techniques that assess the disagreement between two models: a generating model and a verifying model.
CMP measures the surprise of the verifying model at the generated tokens, while CME quantifies the uncertainty at those positions. Both methods require no additional training and can be integrated into existing systems seamlessly.
In experiments, CMP achieved a mean area under the receiver operating characteristic curve (AUROC) of 0.75 on the MMLU benchmark, significantly outperforming the baseline of 0.59 from within-model entropy. These findings suggest that cross-model disagreement can enhance deployment monitoring and improve the reliability of language models in real-world applications.