Abstract:Natural-language explanations are often treated as a unified interface for understanding model behavior, but different explanation sources may support simulation in different ways. This paper compares two families of explanations for question answering models: verbalized feature attributions and self-generated rationales. We evaluate them under a shared counterfactual simulation setting, using an LLM judge as predictor and measuring whether it can better predict a model's answers to follow-up questions when given its explanation. Across multiple instruction-tuned models, we analyze how explanation source, verbalization strategy, and feature granularity affect the simulatability of explanations. Our results show that explanation format and granularity affect simulatability: attribution-based explanations and self-generated rationales differ in how much they improve counterfactual prediction, with effects that vary across models and formats.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2606.01148 [cs.CL] |
| (or arXiv:2606.01148v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.01148 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Pingjun Hong [view email]
[v1]
Sun, 31 May 2026 10:35:35 UTC (401 KB)