Fully Gap-Dependent Bounds for Multinomial Logit Bandit

TL;DR

This paper introduces new algorithms for the multinomial logit bandit problem that achieve fully gap-dependent bounds, improving the understanding of how suboptimality gaps influence learning efficiency and regret.

Contribution

It presents the first algorithms with gap-dependent bounds for MNL bandits, relating the problem to top-K arm identification and introducing novel epoch-based and layer-based estimation techniques.

Findings

Algorithms identify optimal assortments efficiently with high probability.

Regret bounds depend explicitly on suboptimality gaps of items.

First to achieve fully gap-dependent bounds in MNL bandit setting.

Abstract

We study the multinomial logit (MNL) bandit problem, where at each time step, the seller offers an assortment of size at most $K$ from a pool of $N$ items, and the buyer purchases an item from the assortment according to a MNL choice model. The objective is to learn the model parameters and maximize the expected revenue. We present (i) an algorithm that identifies the optimal assortment $S^*$ within $\widetilde{O}(\sum_{i = 1}^N \Delta_i^{-2})$ time steps with high probability, and (ii) an algorithm that incurs $O(\sum_{i \notin S^*} K\Delta_i^{-1} \log T)$ regret in $T$ time steps. To our knowledge, our algorithms are the first to achieve gap-dependent bounds that fully depends on the suboptimality gaps of all items. Our technical contributions include an algorithmic framework that relates the MNL-bandit problem to a variant of the top-$K$ arm identification problem in multi-armed bandits, a generalized epoch-based offering procedure, and a layer-based adaptive estimation procedure.