Abstract:Understanding internal representations of neural models is a core interest of mechanistic interpretability. Due to its large dimensionality, the representation space can encode various aspects about inputs. To what extent are different aspects organized and encoded in separate subspaces? Is it possible to find these ``natural'' subspaces in a purely unsupervised way? Somewhat surprisingly, we can indeed achieve this and find interpretable subspaces by a seemingly unrelated training objective. Our method, neighbor distance minimization (NDM), learns non-basis-aligned subspaces in an unsupervised manner. Qualitative analysis shows subspaces are interpretable in many cases, and encoded information in obtained subspaces tends to share the same abstract concept across different inputs, making such subspaces similar to ``variables'' used by the model. We also conduct quantitative experiments using known circuits in GPT-2; results show a strong connection between subspaces and circuit variables. We also provide evidence showing scalability to 2B models by finding separate subspaces mediating context and parametric knowledge routing. Viewed more broadly, our findings offer a new perspective on understanding model internals and building circuits.
| Comments: | Published as a conference paper at ICLR 2026 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2508.01916 [cs.LG] |
| (or arXiv:2508.01916v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2508.01916 arXiv-issued DOI via DataCite |
Submission history
From: Xinting Huang [view email]
[v1]
Sun, 3 Aug 2025 20:59:29 UTC (20,279 KB)
[v2]
Fri, 20 Feb 2026 12:58:11 UTC (23,281 KB)
[v3]
Thu, 14 May 2026 06:57:50 UTC (23,281 KB)