# Multi-objective Contextual Bandit Problem with Similarity Information

## TL;DR

This paper introduces a multi-objective contextual bandit problem with similarity information, proposing a new algorithm called Pareto Contextual Zooming (PCZ) that efficiently learns near-optimal arms with provable regret bounds.

## Contribution

The paper extends the classical contextual bandit framework to multiple conflicting objectives with similarity information and develops the PCZ algorithm with theoretical regret guarantees.

## Key findings

- PCZ achieves near-optimal Pareto regret bounds.
- Regret scales as  O(T^{(1+d_p)/(2+d_p)}) with the Pareto zooming dimension.
- The regret bound is nearly tight, matching a derived lower bound.

## Abstract

In this paper we propose the multi-objective contextual bandit problem with similarity information. This problem extends the classical contextual bandit problem with similarity information by introducing multiple and possibly conflicting objectives. Since the best arm in each objective can be different given the context, learning the best arm based on a single objective can jeopardize the rewards obtained from the other objectives. In order to evaluate the performance of the learner in this setup, we use a performance metric called the contextual Pareto regret. Essentially, the contextual Pareto regret is the sum of the distances of the arms chosen by the learner to the context dependent Pareto front. For this problem, we develop a new online learning algorithm called Pareto Contextual Zooming (PCZ), which exploits the idea of contextual zooming to learn the arms that are close to the Pareto front for each observed context by adaptively partitioning the joint context-arm set according to the observed rewards and locations of the context-arm pairs selected in the past. Then, we prove that PCZ achieves $\tilde O (T^{(1+d_p)/(2+d_p)})$ Pareto regret where $d_p$ is the Pareto zooming dimension that depends on the size of the set of near-optimal context-arm pairs. Moreover, we show that this regret bound is nearly optimal by providing an almost matching $\Omega (T^{(1+d_p)/(2+d_p)})$ lower bound.