Off-Policy Evaluation with Out-of-Sample Guarantees

TL;DR

This paper introduces a sample-splitting approach for off-policy evaluation that provides finite-sample guarantees on the entire loss distribution, accounting for model misspecification and unmeasured confounding.

Contribution

It presents a novel method for valid out-of-sample policy performance inference with finite-sample guarantees, even under model misspecification.

Findings

Finite-sample coverage guarantees for loss distribution

Method accounts for unmeasured confounding

Applicable under a range of model assumptions

Abstract

We consider the problem of evaluating the performance of a decision policy using past observational data. The outcome of a policy is measured in terms of a loss (aka. disutility or negative reward) and the main problem is making valid inferences about its out-of-sample loss when the past data was observed under a different and possibly unknown policy. Using a sample-splitting method, we show that it is possible to draw such inferences with finite-sample coverage guarantees about the entire loss distribution, rather than just its mean. Importantly, the method takes into account model misspecifications of the past policy - including unmeasured confounding. The evaluation method can be used to certify the performance of a policy using observational data under a specified range of credible model assumptions.