Off-Policy Evaluation for Recommendations with Missing-Not-At-Random Rewards

TL;DR

This paper addresses the challenge of unbiased off-policy evaluation in recommender systems when rewards are missing not at random, proposing a new estimator that effectively mitigates multiple biases and improves evaluation accuracy.

Contribution

The paper introduces a novel estimator leveraging dual propensity scores to reduce bias in off-policy evaluation with missing-not-at-random rewards.

Findings

Proposed estimator outperforms existing methods under high bias conditions.

The estimator effectively mitigates position bias and missing-not-at-random reward bias.

Experimental results demonstrate improved accuracy in off-policy evaluation.

Abstract

Unbiased recommender learning (URL) and off-policy evaluation/learning (OPE/L) techniques are effective in addressing the data bias caused by display position and logging policies, thereby consistently improving the performance of recommendations. However, when both bias exits in the logged data, these estimators may suffer from significant bias. In this study, we first analyze the position bias of the OPE estimator when rewards are missing not at random. To mitigate both biases, we propose a novel estimator that leverages two probabilities of logging policies and reward observations as propensity scores. Our experiments demonstrate that the proposed estimator achieves superior performance compared to other estimators, even as the levels of bias in reward observations increases.