Vector Optimization with Stochastic Bandit Feedback

TL;DR

This paper extends vector optimization to stochastic bandit feedback, characterizing sample complexity for identifying approximate Pareto sets using a new cone-dependent complexity measure, with theoretical bounds and experimental validation.

Contribution

It introduces a generalized framework for vector optimization with bandit feedback, defining ordering complexity and analyzing sample complexity bounds for Pareto set identification.

Findings

Sample complexity scales quadratically with ordering complexity.

Naive elimination algorithm nearly matches worst-case bounds.

Experiments confirm theoretical predictions and show impact of parameters.

Abstract

We introduce vector optimization problems with stochastic bandit feedback, in which preferences among designs are encoded by a polyhedral ordering cone $C$. Our setup generalizes the best arm identification problem to vector-valued rewards by extending the concept of Pareto set beyond multi-objective optimization. We characterize the sample complexity of ($\epsilon,\delta$)-PAC Pareto set identification by defining a new cone-dependent notion of complexity, called the ordering complexity. In particular, we provide gap-dependent and worst-case lower bounds on the sample complexity and show that, in the worst-case, the sample complexity scales with the square of ordering complexity. Furthermore, we investigate the sample complexity of the na\"ive elimination algorithm and prove that it nearly matches the worst-case sample complexity. Finally, we run experiments to verify our theoretical results and illustrate how $C$ and sampling budget affect the Pareto set, the returned ($\epsilon,\delta$)-PAC Pareto set, and the success of identification.