IntOPE: Off-Policy Evaluation in the Presence of Interference

TL;DR

This paper introduces IntIPW, a new off-policy evaluation method that accounts for interference among individuals, addressing a key limitation of traditional approaches that assume independent rewards.

Contribution

The paper proposes IntIPW, an IPW-style estimator that incorporates interference effects into off-policy evaluation, extending existing methods to more realistic scenarios.

Findings

IntIPW outperforms traditional OPE methods in interference settings.

Extensive experiments validate the effectiveness of IntIPW on synthetic and real data.

The method provides more accurate policy evaluation when interference is present.

Abstract

Off-Policy Evaluation (OPE) is employed to assess the potential impact of a hypothetical policy using logged contextual bandit feedback, which is crucial in areas such as personalized medicine and recommender systems, where online interactions are associated with significant risks and costs. Traditionally, OPE methods rely on the Stable Unit Treatment Value Assumption (SUTVA), which assumes that the reward for any given individual is unaffected by the actions of others. However, this assumption often fails in real-world scenarios due to the presence of interference, where an individual's reward is affected not just by their own actions but also by the actions of their peers. This realization reveals significant limitations of existing OPE methods in real-world applications. To address this limitation, we propose IntIPW, an IPW-style estimator that extends the Inverse Probability Weighting (IPW) framework by integrating marginalized importance weights to account for both individual actions and the influence of adjacent entities. Extensive experiments are conducted on both synthetic and real-world data to demonstrate the effectiveness of the proposed IntIPW method.