Off-policy evaluation beyond overlap: partial identification through smoothness

TL;DR

This paper develops a method for off-policy evaluation that provides bounds on policy value without overlap assumptions, using Lipschitz smoothness to formulate and solve linear programs efficiently.

Contribution

It introduces a novel partial identification approach for OPE under non-parametric smoothness assumptions, with closed-form solutions and optimal convergence rates.

Findings

Bounds are tighter with smoothness assumptions.

Linear programs can be solved efficiently.

Method achieves minimax optimal convergence rates.

Abstract

Off-policy evaluation (OPE) is the problem of estimating the value of a target policy using historical data collected under a different logging policy. OPE methods typically assume overlap between the target and logging policy, enabling solutions based on importance weighting and/or imputation. In this work, we approach OPE without assuming either overlap or a well-specified model by considering a strategy based on partial identification under non-parametric assumptions on the conditional mean function, focusing especially on Lipschitz smoothness. Under such smoothness assumptions, we formulate a pair of linear programs whose optimal values upper and lower bound the contributions of the no-overlap region to the off-policy value. We show that these linear programs have a concise closed form solution that can be computed efficiently and that their solutions converge, under the Lipschitz assumption, to the sharp partial identification bounds on the off-policy value. Furthermore, we show that the rate of convergence is minimax optimal, up to log factors. We deploy our methods on two semi-synthetic examples, and obtain informative and valid bounds that are tighter than those possible without smoothness assumptions.