Distributional Off-Policy Evaluation for Slate Recommendations

TL;DR

This paper introduces a new estimator for the complete off-policy performance distribution in slate recommendation systems, enabling more comprehensive evaluation along risk and fairness axes.

Contribution

It develops an unbiased, consistent estimator for the full performance distribution of slate recommendation strategies, extending prior off-policy evaluation methods.

Findings

Significant variance reduction in estimates

Improved sample efficiency over previous methods

Validated on synthetic and real-world data

Abstract

Recommendation strategies are typically evaluated by using previously logged data, employing off-policy evaluation methods to estimate their expected performance. However, for strategies that present users with slates of multiple items, the resulting combinatorial action space renders many of these methods impractical. Prior work has developed estimators that leverage the structure in slates to estimate the expected off-policy performance, but the estimation of the entire performance distribution remains elusive. Estimating the complete distribution allows for a more comprehensive evaluation of recommendation strategies, particularly along the axes of risk and fairness that employ metrics computable from the distribution. In this paper, we propose an estimator for the complete off-policy performance distribution for slates and establish conditions under which the estimator is unbiased and consistent. This builds upon prior work on off-policy evaluation for slates and off-policy distribution estimation in reinforcement learning. We validate the efficacy of our method empirically on synthetic data as well as on a slate recommendation simulator constructed from real-world data (MovieLens-20M). Our results show a significant reduction in estimation variance and improved sample efficiency over prior work across a range of slate structures.