Abstract:How do LLMs learn in-context? Is it by pattern-matching recent tokens, or by inferring latent structure? We probe this question using a toy graph random-walk across two competing graph structures. This task's answer is, in principle, decidable: either the model tracks global topology, or it copies local transitions. We present two lines of evidence that neither account alone is sufficient. First, reconstructing the internal representation structure via PCA reveals that at intermediate mixture ratios, both graph topologies are encoded in orthogonal principal subspaces simultaneously. This pattern is difficult to reconcile with purely local transition copying. Second, residual-stream activation patching and graph-difference steering causally intervene on this graph-family signal: late-layer patching almost fully transfers the clean graph preference, while linear steering moves predictions in the intended direction and fails under norm-matched and label-shuffled controls. Taken together, our findings are most consistent with a dual-mechanism account in which genuine structure inference and induction circuits operate in parallel.
| Comments: | Under review at ICML Mechanistic Interpretability Workshop 2026 |
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.08405 [cs.AI] |
| (or arXiv:2605.08405v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.08405 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Katharine Kowalyshyn [view email]
[v1]
Fri, 8 May 2026 19:11:19 UTC (1,243 KB)