Reimagining service delivery in the agentic era with Google Public Sector
cloud.google.com·1d ago
AI Research
arXiv CS.AI·May 26·5 min read
Polymorphism Is Rotation: Operational Mechanistic Interpretability from a Two-Layer Transformer to Pythia-70m
TL;DR
Transformers trained independently can produce the same outputs despite differing internal representations, a phenomenon termed polymorphism. A method using orthogonal Procrustes fitting allows for the transfer of feature dictionaries between models without retraining.
✦ Why It Matters
Engineers can leverage this rotation technique to enhance model interpretability and transferability without retraining.
Key Takeaways
- Explore polymorphism to understand model behavior across different training seeds.
- Use orthogonal Procrustes fitting to align model representations effectively.
- Achieve high cosine similarity between decoder columns across seeds.
- Enhance interpretability of large-scale models with minimal computational overhead.
How It Works
Polymorphism is characterized by the same function being computed by models with different internal representations. The orthogonal Procrustes fit aligns these representations by applying a single matrix multiplication, which allows for the transfer of feature dictionaries and steering vectors between models.
This method effectively mitigates the differences caused by random rotations in the model's internal space, enabling a clearer understanding of how models process information.