Abstract:The Average Gradient Outer Product (AGOP) governs feature learning in neural networks: the Neural Feature Ansatz states that weight Gram matrices at each layer align with the corresponding AGOP matrices computed over the training distribution. We ask a complementary question: can this same quantity serve as a post-hoc attribution method for explaining individual predictions? We introduce AGOP-Weighted: a novel attribution method that multiplies the per-sample gradient by sqrt(diag(M) / max diag(M)), a training-distribution prior that suppresses gradient noise and amplifies consistently important pixels -- a combination not present in any prior attribution method. We formalise two companion variants -- AGOP-Local (per-sample gradient, equivalent to VanillaGrad) and AGOP-Global (diag(M) directly as a zero-cost saliency map) -- and implement an efficient training-time accumulation hook; AGOP-Global then requires zero inference cost (disk lookup) while AGOP-Weighted requires only a single gradient pass. We conduct the first rigorous comparison of AGOP attribution against Integrated Gradients (IG), SmoothGrad, GradCAM, and VanillaGrad across two benchmarks with pixel-level ground truth: (i) the synthetic XAI-TRIS benchmark (four classification scenarios, 8x8 images, CNN8by8) and (ii) the photorealistic CLEVR-XAI benchmark (ResNet-18 fine-tuned from ImageNet). AGOP-Weighted achieves 44% higher mIoU than IG on linear tasks; AGOP-Global achieves 7x higher mIoU than IG on multiplicative tasks (where IG falls below random) at zero inference cost. Both findings generalise to ResNet-18 on CLEVR-XAI (+18% and +37% respectively). We further show that GradCAM fails on small-resolution images due to spatial resolution collapse, and that diag(M) quality improves monotonically throughout training even after classification accuracy has plateaued.
| Comments: | 8 pages. Accepted at the 4th World Conference on eXplainable Artificial Intelligence (XAI 2026), Late-Breaking Work track, Fortaleza, Brazil, July 1-3, 2026 |
| Subjects: | Machine Learning (cs.LG) |
| ACM classes: | I.2.6; I.2.10 |
| Cite as: | arXiv:2605.12816 [cs.LG] |
| (or arXiv:2605.12816v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.12816 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Raj Kiran Gupta Katakam [view email]
[v1]
Tue, 12 May 2026 23:15:47 UTC (37 KB)