Safe Exploration for Efficient Policy Evaluation and Comparison

TL;DR

This paper studies how to efficiently and safely collect data for accurate bandit policy evaluation, proposing new algorithms with theoretical guarantees and experimental validation.

Contribution

It formulates the problem of safe data collection for policy evaluation, analyzes its properties, and develops novel algorithms for optimal exploration.

Findings

Proposed exploration policies improve evaluation accuracy.

Algorithms are supported by theoretical analysis.

Experimental results validate effectiveness.

Abstract

High-quality data plays a central role in ensuring the accuracy of policy evaluation. This paper initiates the study of efficient and safe data collection for bandit policy evaluation. We formulate the problem and investigate its several representative variants. For each variant, we analyze its statistical properties, derive the corresponding exploration policy, and design an efficient algorithm for computing it. Both theoretical analysis and experiments support the usefulness of the proposed methods.