NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·May 25·12 min read
The Deterministic Horizon: Impossibility Results as Design Specifications for Trustworthy AI Systems
TL;DR
Large language models face fundamental computational limits rooted in mathematical impossibility theorems, yet these constraints remain largely ignored in practice. This work introduces the Deterministic Horizon—a computable architectural ceiling on model accuracy derived from residual stream capacity—and converts sixteen impossibility results into concrete design specifications.
✦ Why It Matters
Engineers can compute accuracy ceilings before training, allocate compute efficiently, and design systems respecting fundamental limits rather than chasing impossible improvements.
Key Takeaways
- Understand the Deterministic Horizon to set realistic model expectations.
- Use quantifiable boundaries to guide AI system design.
- Apply findings to improve preference learning and auction mechanisms.
- Integrate impossibility results into your development workflow.
How It Works
The Deterministic Horizon is determined by the architecture of transformer models, specifically the number of layers and embedding width. Beyond a critical depth, no training adjustments can improve accuracy, leading to a super-exponential decay in performance.
This insight allows developers to compute expected model limits before deployment, ensuring more informed design choices.