Tractable Multinomial Logit Contextual Bandits with Non-Linear Utilities

TL;DR

This paper introduces a computationally efficient algorithm for multinomial logit contextual bandits with non-linear utility functions, including neural networks, achieving near-optimal regret bounds and demonstrating strong empirical performance.

Contribution

It presents the first tractable algorithm for MNL contextual bandits with non-linear utilities that provably attains  ilde{O}(\u007F\u007F ext{sqrt}(T)) regret, extending beyond linear utility models.

Findings

Achieves  ilde{O}(\u007F ext ext{sqrt}(T)) regret bound.

Effective in both realizable and misspecified scenarios.

First computationally tractable method with provable guarantees for non-linear utilities.

Abstract

We study the multinomial logit (MNL) contextual bandit problem for sequential assortment selection. Although most existing research assumes utility functions to be linear in item features, this linearity assumption restricts the modeling of intricate interactions between items and user preferences. A recent work (Zhang & Luo, 2024) has investigated general utility function classes, yet its method faces fundamental trade-offs between computational tractability and statistical efficiency. To address this limitation, we propose a computationally efficient algorithm for MNL contextual bandits leveraging the upper confidence bound principle, specifically designed for non-linear parametric utility functions, including those modeled by neural networks. Under a realizability assumption and a mild geometric condition on the utility function class, our algorithm achieves a regret bound of $\tilde{O}(\sqrt{T})$, where $T$ denotes the total number of rounds. Our result establishes that sharp $\tilde{O}(\sqrt{T})$-regret is attainable even with neural network-based utilities, without relying on strong assumptions such as neural tangent kernel approximations. To the best of our knowledge, our proposed method is the first computationally tractable algorithm for MNL contextual bandits with non-linear utilities that provably attains $\tilde{O}(\sqrt{T})$ regret. Comprehensive numerical experiments validate the effectiveness of our approach, showing robust performance not only in realizable settings but also in scenarios with model misspecification.