Asymptotic Inference for Multi-Stage Stationary Treatment Policy with Variable Selection

TL;DR

This paper develops an asymptotic inference framework for multi-stage stationary treatment policies with variable selection, addressing high-dimensional features and providing valid statistical inference for policy parameters.

Contribution

It introduces a novel estimation and inference method for multi-stage stationary policies with feature selection in high-dimensional settings.

Findings

The proposed estimator achieves asymptotic normality even with slow convergence of nuisance parameters.

The method effectively identifies important features and estimates near-optimal policies.

Numerical studies confirm valid inference and good policy performance.

Abstract

Dynamic treatment regimes or policies are a sequence of decision functions over multiple stages that are tailored to individual features. One important class of treatment policies in practice, namely multi-stage stationary treatment policies, prescribes treatment assignment probabilities using the same decision function across stages, where the decision is based on the same set of features consisting of time-evolving variables (e.g., routinely collected disease biomarkers). Although there has been extensive literature on constructing valid inference for the value function associated with dynamic treatment policies, little work has focused on the policies themselves, especially in the presence of high-dimensional feature variables. We aim to fill the gap in this work. Specifically, we first estimate the multi-stage stationary treatment policy using an augmented inverse probability weighted estimator for the value function to increase asymptotic efficiency, and further apply a penalty to select important feature variables. We then construct one-step improvements of the policy parameter estimators for valid inference. Theoretically, we show that the improved estimators are asymptotically normal, even if nuisance parameters are estimated at a slow convergence rate and the dimension of the feature variables increases with the sample size. Our numerical studies demonstrate that the proposed method estimates a sparse policy with a near-optimal value function and conducts valid inference for the policy parameters.