Abstract:Comparative evaluation of multiple dynamic treatment policies is essential for healthcare and policy decisions, yet conventional longitudinal causal inference methods estimate each in isolation, preventing information sharing across counterfactuals. We demonstrate that this separate estimation paradigm induces a structurally uncontrolled second-order bias, inflating finite-sample variance even after standard debiasing with longitudinal targeted maximum likelihood estimation(LTMLE). To address this, we propose a policy-aware reparameterization of Iterative Conditional Expectation (ICE) Q-functions that enables joint estimation through shared representations. We implement this approach in the Policy-Encoded Q Network (PEQ-Net), an architecture centered on a shared policy encoder. The encoder is trained using kernel mean embeddings, ensuring that the learned representation space reflects population-level policy dissimilarities. After applying an LTMLE correction step, we prove this design imposes a structural constraint on the second-order remainder, thereby stabilizing finite-sample variance. Experiments on semi-synthetic datasets demonstrate that PEQ-Net consistently outperforms existing ICE-based methods, achieving substantial reductions in root-mean-square error, particularly when evaluating closely related policies.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.14284 [cs.LG] |
| (or arXiv:2605.14284v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.14284 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wenxin Chen [view email]
[v1]
Thu, 14 May 2026 02:33:58 UTC (1,077 KB)