Modeling Choice via Self-Attention

TL;DR

This paper introduces a novel neural network-based choice model using self-attention, providing theoretical guarantees and demonstrating superior performance on real data compared to existing models.

Contribution

It is the first to integrate self-attention into choice modeling with both theoretical analysis and empirical validation, advancing the intersection of deep learning and operations management.

Findings

The attention-based model is a low-rank generalization of Halo-MNL.

Supports a nonconvex estimator with near-optimal sample complexity.

Outperforms existing models on real-world choice data.

Abstract

Models of choice are a fundamental input to many now-canonical optimization problems in the field of Operations Management, including assortment, inventory, and price optimization. Naturally, accurate estimation of these models from data is a critical step in the application of these optimization problems in practice. Concurrently, recent advancements in deep learning have sparked interest in integrating these techniques into choice modeling. However, there is a noticeable research gap at the intersection of deep learning and choice modeling, particularly with both theoretical and empirical foundations. Thus motivated, we first propose a choice model that is the first to successfully (both theoretically and practically) leverage a modern neural network architectural concept (self-attention). Theoretically, we show that our attention-based choice model is a low-rank generalization of the Halo Multinomial Logit (Halo-MNL) model. We prove that whereas the Halo-MNL requires $\Omega(m^2)$ data samples to estimate, where $m$ is the number of products, our model supports a natural nonconvex estimator (in particular, that which a standard neural network implementation would apply) which admits a near-optimal stationary point with $O(m)$ samples. Additionally, we establish the first realistic-scale benchmark for choice model estimation on real data, conducting the most extensive evaluation of existing models to date, thereby highlighting our model's superior performance.