Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·May 27·5 min read
Composition Collapse: Stable Factual Knowledge Does Not Imply Compositional Reasoning
TL;DR
Standard benchmarks assume models with equal factual knowledge perform equally at multi-hop reasoning (chaining facts across multiple steps). Researchers identified composition collapse: a phenomenon where models with statistically identical atomic knowledge differ by over 40 percentage points in compositional reasoning ability.
✦ Why It Matters
Benchmark scores alone cannot predict compositional reasoning ability; evaluate fact-assembly separately from factual knowledge.
Key Takeaways
- Evaluate compositional reasoning independently from factual knowledge benchmarks.
- 40+ percentage point gaps exist between models with identical atomic knowledge.
- Aggregate scores mask systematic failures in fact assembly and chaining.
- Design benchmarks that explicitly test multi-step fact composition separately.
Full Summary
Current evaluation methods for large language models rely on aggregate benchmark scores that treat multi-hop reasoning—the ability to chain multiple facts together to answer complex questions—as a single unified capability. Researchers discovered composition collapse, a systematic failure mode where models demonstrating statistically indistinguishable performance on individual factual knowledge tasks produce dramatically different results when required to compose those facts together.
The study found performance gaps exceeding 40 percentage points between models with equivalent atomic knowledge scores. This phenomenon exposes a fundamental assumption in model evaluation: that better performance on isolated facts automatically translates to better compositional reasoning.
The findings suggest current benchmarking practices obscure important capability gaps and that factual knowledge and compositional ability are separable, independently variable properties. Engineers and researchers must evaluate composition separately from atomic knowledge to accurately assess model capabilities.