Inference for relative sparsity

TL;DR

This paper develops statistical inference methods for relative sparsity policies in multi-stage healthcare decision models, addressing uncertainty quantification and extending previous single-stage work.

Contribution

It introduces inference techniques for the relative sparsity penalty in multi-stage settings, incorporating a Trust Region Policy Optimization framework and a sample-splitting approach.

Findings

Effective inference methods for relative sparsity in multi-stage policies.

Demonstrated robustness through extensive simulations.

Applied approach to real electronic health record data.

Abstract

In healthcare, there is much interest in estimating policies, or mappings from covariates to treatment decisions. Recently, there is also interest in constraining these estimated policies to the standard of care, which generated the observed data. A relative sparsity penalty was proposed to derive policies that have sparse, explainable differences from the standard of care, facilitating justification of the new policy. However, the developers of this penalty only considered estimation, not inference. Here, we develop inference for the relative sparsity objective function, because characterizing uncertainty is crucial to applications in medicine. Further, in the relative sparsity work, the authors only considered the single-stage decision case; here, we consider the more general, multi-stage case. Inference is difficult, because the relative sparsity objective depends on the unpenalized value function, which is unstable and has infinite estimands in the binary action case. Further, one must deal with a non-differentiable penalty. To tackle these issues, we nest a weighted Trust Region Policy Optimization function within a relative sparsity objective, implement an adaptive relative sparsity penalty, and propose a sample-splitting framework for post-selection inference. We study the asymptotic behavior of our proposed approaches, perform extensive simulations, and analyze a real, electronic health record dataset.