Learning in Random Utility Models Via Online Decision Problems

TL;DR

This paper introduces a gradient-based learning algorithm for Random Utility Models in repeated decision-making, demonstrating its consistency and equivalence to established machine learning algorithms, with applications to economic and game-theoretic scenarios.

Contribution

It develops a novel gradient-based learning method for RUMs, proves its Hannan consistency, and links it to the FTRL algorithm, providing an economic foundation for machine learning techniques.

Findings

Large class of RUMs are Hannan consistent

Gradient-based algorithm is equivalent to FTRL

Applications include recency bias and no-regret learning

Abstract

This paper studies the Random Utility Model (RUM) in a repeated stochastic choice situation, in which the decision maker is imperfectly informed about the payoffs of each available alternative. We develop a gradient-based learning algorithm by embedding the RUM into an online decision problem. We show that a large class of RUMs are Hannan consistent (\citet{Hahn1957}); that is, the average difference between the expected payoffs generated by a RUM and that of the best-fixed policy in hindsight goes to zero as the number of periods increase. In addition, we show that our gradient-based algorithm is equivalent to the Follow the Regularized Leader (FTRL) algorithm, which is widely used in the machine learning literature to model learning in repeated stochastic choice problems. Thus, we provide an economically grounded optimization framework to the FTRL algorithm. Finally, we apply our framework to study recency bias, no-regret learning in normal form games, and prediction markets.