NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·Jun 10·5 min read
Using Probabilistic Programs to Train Inductive Reasoning in Large Language Models
TL;DR
Inductive reasoning in large language models (LLMs) is challenging due to the lack of high-quality labeled datasets and the need for handling uncertain outputs. A new method called Program-based Posterior Training (PPT) generates diverse scenarios as probabilistic programs, allowing LLMs to learn from these distributional targets.
✦ Why It Matters
Engineers can leverage PPT to improve LLMs' performance on tasks requiring inductive reasoning and uncertainty handling.
Key Takeaways
- Explore Program-based Posterior Training for LLMs in your projects.
- Utilize probabilistic programs to generate diverse training scenarios.
- Focus on improving model alignment with human judgments.
- Consider uncertainty handling as a key aspect of model training.
Full Summary
Inductive reasoning, which involves making inferences from incomplete or ambiguous data, poses significant challenges for large language models (LLMs) that typically excel in deductive reasoning tasks. To address these challenges, Program-based Posterior Training (PPT) was developed, which utilizes LLMs to create a variety of open-world scenarios represented as probabilistic programs.
These programs are then used to perform probabilistic inference, generating soft labels that reflect uncertainty. The researchers fine-tuned LLMs on 10,000 such scenarios and evaluated their performance on held-out tasks and external benchmarks.
Results showed substantial improvements in estimation accuracy and better alignment with human judgments, indicating that LLMs trained with PPT have internalized uncertainty more effectively than those using traditional methods. This suggests that probabilistic-program-mediated fine-tuning could enhance LLMs' capabilities in real-world reasoning tasks.