NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·3h ago
AI Research
arXiv CS.AI·Jun 12·10 min read
The Query Channel: Information-Theoretic Limits of Masking-Based Explanations
TL;DR
Masking-based explanation methods like KernelSHAP and LIME struggle with information limits when explaining model predictions. This paper introduces a framework that treats the explanation process as communication over a query channel, revealing the constraints of these methods.
✦ Why It Matters
Engineers can leverage these insights to improve the reliability of model explanations in AI applications.
Key Takeaways
- Understand the communication framework for masking-based explanations.
- Assess the capacity limits of your explanation methods.
- Utilize Monte Carlo estimators for benchmarking query efficiency.
- Consider super-pixel resolution choices to optimize explanation quality.
How It Works
The authors model the explanation process as a communication channel, where the complexity of the explanation is quantified by the entropy of the hypothesis class. Each query to the model under perturbation serves as a channel use, and the information rate is determined by the identification capacity per query.
This framework allows for a rigorous analysis of when reliable feature importance recovery is achievable.