Reimagining service delivery in the agentic era with Google Public Sector
cloud.google.com·19h ago
AI Research
arXiv CS.AI·Jun 9·5 min read
A Geometric Unification of Concept Learning with Concept Cones
TL;DR
A gap exists between Concept Bottleneck Models (CBMs), which define concepts based on human labels, and Sparse Autoencoders (SAEs), which identify emergent concepts through sparse coding. This work demonstrates that both CBMs and SAEs share a common geometric structure by learning linear directions in activation space.
✦ Why It Matters
Engineers can leverage this unified framework to enhance model interpretability and design more effective AI systems.
Key Takeaways
- Explore the geometric structure of concept learning in your models.
- Use the proposed metrics to assess SAE performance.
- Identify optimal sparsity and expansion factors for better alignment.
- Bridge supervised and unsupervised learning approaches in your projects.
How It Works
The study reveals that both CBMs and SAEs learn concept cones in activation space, where CBMs use human-defined concepts to guide learning, while SAEs discover concepts through sparse coding. This shared geometric perspective allows for a systematic evaluation of how well SAEs approximate the concept cones defined by CBMs.