NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·May 25·8 min read
When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions
TL;DR
Chain-of-thought reasoning (explicit step-by-step problem solving) is applied universally to LLMs despite often wasting tokens on tasks where it provides no benefit. Researchers discovered that entropy dynamics—the statistical uncertainty in early token generation—reliably signals when reasoning helps, exhibiting phase-transition-like behavior.
✦ Why It Matters
Engineers can reduce LLM inference costs by 40–55% while maintaining or improving accuracy by selectively applying reasoning based on entropy signals rather than default strategies.
Key Takeaways
- Monitor early-stage entropy patterns to predict whether chain-of-thought reasoning will benefit specific tasks.
- Deploy EDRM as a training-free routing layer to adaptively select between standard and reasoning-enhanced inference.
- Expect 41–55% token savings with minimal calibration data (50 samples) across diverse benchmarks.
- Avoid applying chain-of-thought by default; use entropy dynamics to invoke reasoning selectively per instance.
Full Summary
Chain-of-thought (CoT) prompting has become standard practice for improving large language model performance, yet empirical evidence shows it often provides marginal or negative gains on factual and open-ended tasks while significantly increasing computational cost. This work reframes reasoning not as a static task property but as a dynamic decoding state—a pattern that emerges during token generation.
By analyzing entropy dynamics (how statistical uncertainty evolves in early generation), the authors found that tasks benefiting from CoT show consistent entropy reduction, while others display unstable or increasing patterns, resembling a phase transition from exploratory to structured reasoning. Based on these insights, they developed EDRM (Entropy Dynamics-based Reasoning Manifold), a lightweight, training-free framework that embeds entropy trajectories into a compact manifold representation to adaptively route between inference strategies.
Across 15 benchmarks and 4 LLM architectures, EDRM achieved 41–55% token reduction at dataset level and up to 4.7% accuracy improvement at instance level, using as few as 50 calibration samples.