Reimagining service delivery in the agentic era with Google Public Sector
cloud.google.com·19h ago
AI Research
arXiv CS.AI·Jun 10·5 min read
When Attribution Patching Lies: Diagnosis and a Second-Order Correction
TL;DR
Attribution patching, a method for understanding language model behavior, suffers from reliability issues due to non-linearities in the model. This work introduces a Hessian-vector-product (HVP) correction that significantly reduces errors with minimal computational overhead.
✦ Why It Matters
Engineers can leverage HVP for more reliable model interpretability without incurring high computational costs.
Key Takeaways
- Use HVP correction to improve reliability in model interpretation.
- Evaluate model components flagged as unreliable for targeted analysis.
- Adopt multi-step HVP for better accuracy at lower compute costs.
- Understand non-linearities in your model to enhance interpretability.
How It Works
HVP correction addresses the leading-order errors in attribution patching by leveraging second-order information from the model's Hessian. This method requires only one additional backward pass, making it computationally feasible for larger models.
By focusing on the non-linearities in the downstream network, HVP provides a more accurate estimate of component importance.