Optimal Multi-Objective Best Arm Identification with Fixed Confidence

TL;DR

This paper addresses the challenge of efficiently identifying the best arms across multiple objectives in a multi-armed bandit setting, proposing an asymptotically optimal algorithm that simplifies complex computations and is supported by theoretical and empirical validation.

Contribution

It introduces a novel algorithm using surrogate proportions for multi-objective best arm identification, eliminating the need for computationally expensive optimization at each step.

Findings

The proposed algorithm is asymptotically optimal.

The algorithm outperforms existing methods in empirical tests.

Theoretical analysis confirms the efficiency of the approach.

Abstract

We consider a multi-armed bandit setting with finitely many arms, in which each arm yields an $M$-dimensional vector reward upon selection. We assume that the reward of each dimension (a.k.a. {\em objective}) is generated independently of the others. The best arm of any given objective is the arm with the largest component of mean corresponding to the objective. The end goal is to identify the best arm of {\em every} objective in the shortest (expected) time subject to an upper bound on the probability of error (i.e., fixed-confidence regime). We establish a problem-dependent lower bound on the limiting growth rate of the expected stopping time, in the limit of vanishing error probabilities. This lower bound, we show, is characterised by a max-min optimisation problem that is computationally expensive to solve at each time step. We propose an algorithm that uses the novel idea of {\em surrogate proportions} to sample the arms at each time step, eliminating the need to solve the max-min optimisation problem at each step. We demonstrate theoretically that our algorithm is asymptotically optimal. In addition, we provide extensive empirical studies to substantiate the efficiency of our algorithm. While existing works on pure exploration with multi-objective multi-armed bandits predominantly focus on {\em Pareto frontier identification}, our work fills the gap in the literature by conducting a formal investigation of the multi-objective best arm identification problem.