NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·1h ago
AI Research
arXiv CS.AI·May 21·5 min read
How do LLMs Compute Verbal Confidence
TL;DR
Verbal confidence, which allows large language models (LLMs) to express uncertainty as a numerical score or category, is not well understood. This study investigates when LLMs compute confidence—either on-demand or during response generation—and what this confidence score signifies.
✦ Why It Matters
Understanding LLM confidence computation can improve model reliability and inform better decision-making in AI applications.
Key Takeaways
- Explore how LLMs generate confidence scores during response creation.
- Utilize confidence scores to assess model reliability in applications.
- Consider timing of confidence computation in model design.
- Enhance training methods based on insights into uncertainty representation.
Full Summary
Large language models (LLMs) are increasingly used in applications requiring uncertainty estimates, yet the mechanisms behind their verbal confidence outputs remain unclear. This research explores two key aspects: the timing of confidence computation—whether it occurs just-in-time upon request or is pre-computed during answer generation—and the meaning of the confidence scores provided.
Through experimental analysis, it was found that LLMs typically generate confidence scores during the response creation process, rather than on-demand. Additionally, the study clarifies that these scores represent the model's internal uncertainty regarding its predictions.
The implications of these findings suggest that engineers can better interpret LLM outputs and improve model training by understanding how confidence is derived. This knowledge can enhance the reliability of LLM applications in critical domains.