Thompson Sampling for the MNL-Bandit

TL;DR

This paper introduces a Thompson Sampling approach for the MNL-Bandit problem, a sequential subset selection task with unknown parameters, achieving near-optimal regret and strong numerical results.

Contribution

It adapts Thompson Sampling to the MNL-Bandit problem, providing a near-optimal regret guarantee and demonstrating effective numerical performance.

Findings

Achieves near-optimal regret bounds.

Demonstrates strong numerical performance.

Addresses a broad class of exploration-exploitation problems.

Abstract

We consider a sequential subset selection problem under parameter uncertainty, where at each time step, the decision maker selects a subset of cardinality $K$ from $N$ possible items (arms), and observes a (bandit) feedback in the form of the index of one of the items in said subset, or none. Each item in the index set is ascribed a certain value (reward), and the feedback is governed by a Multinomial Logit (MNL) choice model whose parameters are a priori unknown. The objective of the decision maker is to maximize the expected cumulative rewards over a finite horizon $T$, or alternatively, minimize the regret relative to an oracle that knows the MNL parameters. We refer to this as the MNL-Bandit problem. This problem is representative of a larger family of exploration-exploitation problems that involve a combinatorial objective, and arise in several important application domains. We present an approach to adapt Thompson Sampling to this problem and show that it achieves near-optimal regret as well as attractive numerical performance.