Beyond the Lower Bound: Bridging Regret Minimization and Best Arm Identification in Lexicographic Bandits

TL;DR

This paper introduces algorithms for lexicographic bandits that effectively combine regret minimization and best arm identification, leveraging cross-objective information to outperform traditional bounds and improve decision-making in hierarchical multi-objective settings.

Contribution

It presents two novel elimination-based algorithms that unify regret minimization and best arm identification in lexicographic bandits, with one exploiting cross-objective dependencies for enhanced performance.

Findings

Algorithms achieve comparable bounds to single-objective bandits.

Cross-objective information sharing surpasses known lower bounds.

Empirical results show superior performance over baselines.

Abstract

In multi-objective decision-making with hierarchical preferences, lexicographic bandits provide a natural framework for optimizing multiple objectives in a prioritized order. In this setting, a learner repeatedly selects arms and observes reward vectors, aiming to maximize the reward for the highest-priority objective, then the next, and so on. While previous studies have primarily focused on regret minimization, this work bridges the gap between \textit{regret minimization} and \textit{best arm identification} under lexicographic preferences. We propose two elimination-based algorithms to address this joint objective. The first algorithm eliminates suboptimal arms sequentially, layer by layer, in accordance with the objective priorities, and achieves sample complexity and regret bounds comparable to those of the best single-objective algorithms. The second algorithm simultaneously leverages reward information from all objectives in each round, effectively exploiting cross-objective dependencies. Remarkably, it outperforms the known lower bound for the single-objective bandit problem, highlighting the benefit of cross-objective information sharing in the multi-objective setting. Empirical results further validate their superior performance over baselines.