Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·Jun 9·5 min read
Query Lens: Interpreting Sparse Key-Value Features with Indirect Effects
TL;DR
Sparse autoencoders, while more interpretable than individual neurons, struggle with reliable feature characterization. Query Lens is introduced as an extension of Logit Lens, allowing for better interpretation of sparse features by analyzing both key inputs and value outputs.
✦ Why It Matters
Engineers can use Query Lens to improve the interpretability of their sparse autoencoder models.
Key Takeaways
- Use Query Lens for better feature interpretability in models.
- Identify both inputs and outputs of sparse features effectively.
- Explore indirect effects of features through downstream modules.
- Consider the Subspace Channel Hypothesis for deeper insights.
How It Works
Query Lens enhances feature interpretation by analyzing both key features from the encoder and value features from the decoder. It tracks how features activate based on inputs and how they influence outputs, including indirect effects that occur through subsequent processing layers.
This dual analysis allows for a more nuanced understanding of feature behavior.