NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·1h ago
AI Research
arXiv CS.AI·May 25·8 min read
Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoning
TL;DR
Prior work showed large language models develop similar internal representations despite different training, but whether they share reasoning strategies remained unclear. Researchers compared 16 models across 8 families on 800 reasoning problems using Centered Kernel Alignment (CKA), a metric measuring representation similarity.
✦ Why It Matters
Engineers should reconsider ensemble strategies and interpretability transfer assumptions; model agreement on representations does not guarantee agreement on reasoning.
Key Takeaways
- Evaluate model ensembles for reasoning, not just representation.
- Use CKA metrics to assess model alignment on specific tasks.
- Consider the generation gap in model outputs for better predictions.
- Explore implications of shared information on model interpretability.
How It Works
The study employs Centered Kernel Alignment (CKA) to measure representational similarity among models. CKA quantifies how closely the internal representations of different models align, providing insights into their shared processing strategies.