Robust Offline Policy Learning with Observational Data from Multiple Sources

TL;DR

This paper introduces a robust offline policy learning method that leverages multiple observational data sources to create personalized decision policies with low worst-case regret across diverse target environments.

Contribution

It proposes a minimax regret optimization framework combined with doubly robust evaluation and no-regret algorithms for robust policy learning from heterogeneous data sources.

Findings

Achieves minimal worst-case mixture regret.

Demonstrates improved robustness in policy generalization.

Validates approach with experimental results.

Abstract

We consider the problem of using observational bandit feedback data from multiple heterogeneous data sources to learn a personalized decision policy that robustly generalizes across diverse target settings. To achieve this, we propose a minimax regret optimization objective to ensure uniformly low regret under general mixtures of the source distributions. We develop a policy learning algorithm tailored to this objective, combining doubly robust offline policy evaluation techniques and no-regret learning algorithms for minimax optimization. Our regret analysis shows that this approach achieves the minimal worst-case mixture regret up to a moderated vanishing rate of the total data across all sources. Our analysis, extensions, and experimental results demonstrate the benefits of this approach for learning robust decision policies from multiple data sources.