Off-Policy Evaluation with Policy-Dependent Optimization Response

TL;DR

This paper introduces a novel framework for off-policy evaluation that accounts for policy-dependent optimization responses, addressing bias issues and enabling causal policy optimization in decision-making scenarios.

Contribution

It develops unbiased estimators for policy-dependent causal outcomes and proposes a general algorithm for optimizing causal interventions.

Findings

Constructed unbiased estimators for policy-dependent estimands.

Analyzed asymptotic variance properties of estimators.

Validated theoretical results with numerical simulations.

Abstract

The intersection of causal inference and machine learning for decision-making is rapidly expanding, but the default decision criterion remains an \textit{average} of individual causal outcomes across a population. In practice, various operational restrictions ensure that a decision-maker's utility is not realized as an \textit{average} but rather as an \textit{output} of a downstream decision-making problem (such as matching, assignment, network flow, minimizing predictive risk). In this work, we develop a new framework for off-policy evaluation with \textit{policy-dependent} linear optimization responses: causal outcomes introduce stochasticity in objective function coefficients. Under this framework, a decision-maker's utility depends on the policy-dependent optimization, which introduces a fundamental challenge of \textit{optimization} bias even for the case of policy evaluation. We construct unbiased estimators for the policy-dependent estimand by a perturbation method, and discuss asymptotic variance properties for a set of adjusted plug-in estimators. Lastly, attaining unbiased policy evaluation allows for policy optimization: we provide a general algorithm for optimizing causal interventions. We corroborate our theoretical results with numerical simulations.