Smooth Multi-Policy Causal Effect Estimation in Longitudinal Settings

TL;DR

This paper introduces PEQ-Net, a novel joint estimation method for multiple dynamic treatment policies that reduces variance and bias in longitudinal causal inference.

Contribution

It proposes a policy-aware reparameterization of ICE Q-functions with shared representations, improving estimation accuracy over existing methods.

Findings

PEQ-Net outperforms existing ICE-based methods in semi-synthetic experiments.

It achieves significant reductions in root-mean-square error.

The approach stabilizes finite-sample variance after LTMLE correction.

Abstract

Comparative evaluation of multiple dynamic treatment policies is essential for healthcare and policy decisions, yet conventional longitudinal causal inference methods estimate each in isolation, preventing information sharing across counterfactuals. We demonstrate that this separate estimation paradigm induces a structurally uncontrolled second-order bias, inflating finite-sample variance even after standard debiasing with longitudinal targeted maximum likelihood estimation(LTMLE). To address this, we propose a policy-aware reparameterization of Iterative Conditional Expectation (ICE) Q-functions that enables joint estimation through shared representations. We implement this approach in the Policy-Encoded Q Network (PEQ-Net), an architecture centered on a shared policy encoder. The encoder is trained using kernel mean embeddings, ensuring that the learned representation space reflects population-level policy dissimilarities. After applying an LTMLE correction step, we prove this design imposes a structural constraint on the second-order remainder, thereby stabilizing finite-sample variance. Experiments on semi-synthetic datasets demonstrate that PEQ-Net consistently outperforms existing ICE-based methods, achieving substantial reductions in root-mean-square error, particularly when evaluating closely related policies.