Achieving Limited Adaptivity for Multinomial Logistic Bandits

TL;DR

This paper introduces two algorithms for multinomial logistic bandits that operate with limited policy updates, achieving near-optimal regret and demonstrating practical efficiency in both stochastic and adversarial contexts.

Contribution

The paper proposes B-MNL-CB and RS-MNL algorithms that function with limited adaptivity, extending optimal design concepts and achieving low regret with few policy updates.

Findings

Algorithms achieve () ilde{O}(\u00f7b7f7) regret with limited updates

B-MNL-CB extends distributional optimal designs to multinomial settings

RS-MNL performs well with adversarial contexts and few updates

Abstract

Multinomial Logistic Bandits have recently attracted much attention due to their ability to model problems with multiple outcomes. In this setting, each decision is associated with many possible outcomes, modeled using a multinomial logit function. Several recent works on multinomial logistic bandits have simultaneously achieved optimal regret and computational efficiency. However, motivated by real-world challenges and practicality, there is a need to develop algorithms with limited adaptivity, wherein we are allowed only $M$ policy updates. To address these challenges, we present two algorithms, B-MNL-CB and RS-MNL, that operate in the batched and rarely-switching paradigms, respectively. The batched setting involves choosing the $M$ policy update rounds at the start of the algorithm, while the rarely-switching setting can choose these $M$ policy update rounds in an adaptive fashion. Our first algorithm, B-MNL-CB extends the notion of distributional optimal designs to the multinomial setting and achieves $\tilde{O}(\sqrt{T})$ regret assuming the contexts are generated stochastically when presented with $\Omega(\log \log T)$ update rounds. Our second algorithm, RS-MNL works with adversarially generated contexts and can achieve $\tilde{O}(\sqrt{T})$ regret with $\tilde{O}(\log T)$ policy updates. Further, we conducted experiments that demonstrate that our algorithms (with a fixed number of policy updates) are extremely competitive (and often better) than several state-of-the-art baselines (which update their policy every round), showcasing the applicability of our algorithms in various practical scenarios.