NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·May 28·8 min read
The Computational Boundary of Inference: Capability Internalization, Training, and the Turing Jump
TL;DR
Current AI systems struggle with tasks requiring reasoning beyond their training data—a gap between what models can memorize versus what they can compute. This work explores the theoretical boundary between inference-time computation and training-time capability internalization, connecting it to the Turing Jump (a concept from computability theory about limits of algorithmic power).
✦ Why It Matters
Engineers can use these theoretical boundaries to decide whether to retrain models or rely on inference-time computation for new capabilities.
Key Takeaways
- Some AI capabilities require training to internalize; inference alone cannot compute them.
- Use Turing Jump framework to classify which tasks need retraining versus inference-time solving.
- Inference-time computation has hard limits—identify them before investing in prompt engineering.
- Design training pipelines to internalize capabilities that inference cannot dynamically compute.
Full Summary
AI models today face a critical limitation: they can only reliably perform tasks their training exposed them to, yet inference (using a trained model) theoretically allows some computation beyond memorized patterns. This paper investigates where that boundary lies by examining capability internalization—the process of encoding problem-solving abilities into model weights during training—versus dynamic computation during inference.
The authors connect this to the Turing Jump, a foundational concept in computability theory describing the hierarchy of problems solvable by increasingly powerful computational systems. They analyze how much additional reasoning work inference can perform and identify theoretical constraints on what no amount of inference-time computation can overcome without retraining.
The findings suggest certain problem classes require explicit training to internalize, while others remain computable through inference alone. This has direct implications for designing efficient AI systems: some capabilities must be baked into training, while others can be deferred to inference.