Automated Off-Policy Estimator Selection via Supervised Learning

TL;DR

This paper introduces a supervised learning approach to automatically select the most suitable off-policy evaluation estimator for a given dataset, improving accuracy and reducing computational cost.

Contribution

The paper presents a novel data-driven method that trains a machine learning model to predict the best estimator for off-policy evaluation tasks, addressing a key gap in the literature.

Findings

Outperforms baseline estimator selection methods on real datasets

Reduces computational cost compared to existing approaches

Effectively predicts the optimal estimator for diverse OPE problems

Abstract

The Off-Policy Evaluation (OPE) problem consists of evaluating the performance of counterfactual policies with data collected by another one. To solve the OPE problem, we resort to estimators, which aim to estimate in the most accurate way possible the performance that the counterfactual policies would have had if they were deployed in place of the logging policy. In the literature, several estimators have been developed, all with different characteristics and theoretical guarantees. Therefore, there is no dominant estimator and each estimator may be the best for different OPE problems, depending on the characteristics of the dataset at hand. Although the selection of the estimator is a crucial choice for an accurate OPE, this problem has been widely overlooked in the literature. We propose an automated data-driven OPE estimator selection method based on supervised learning. In particular, the core idea we propose in this paper is to create several synthetic OPE tasks and use a machine learning model trained to predict the best estimator for those synthetic tasks. We empirically show how our method is able to perform a better estimator selection compared to a baseline method on several real-world datasets, with a computational cost significantly lower than the one of the baseline.