Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·May 28·8 min read
Localizing Input Uncertainty Quantification for Large Language Models via Shapley Values
TL;DR
Large language models (LLMs) lack reliable methods to identify which input tokens contribute most to prediction uncertainty. Researchers developed a Shapley value-based approach that localizes uncertainty quantification—pinpointing specific input positions responsible for model confidence variations.
✦ Why It Matters
Engineers can now identify which input tokens cause model uncertainty, enabling targeted prompt refinement and failure diagnosis.
Key Takeaways
- Utilize ShaQ for detailed input uncertainty analysis.
- Identify specific input spans that contribute to uncertainty.
- Enhance decision-making reliability in AI applications.
- Apply findings to improve user interactions with LLMs.
How It Works
ShaQ models input spans as players in a cooperative game, using Shapley values to assess their contributions to uncertainty. By calculating the marginal reductions in conditional entropy when clarifying different combinations of spans, it provides a comprehensive view of how each part of the input affects overall uncertainty.