On a PDE model for Learning in Stochastic Market Entry Games

TL;DR

This paper develops a PDE-based continuum model for stochastic reinforcement learning in market entry games, capturing key phenomena like aggregate learning and sorting, with proven solution properties and explicit time scales.

Contribution

It derives a nonlinear PDE from microscopic rules, proving well-posedness and analyzing long-term behavior, linking theoretical insights with observed market phenomena.

Findings

The PDE models aggregate learning and sorting in market entry.

Aggregate learning occurs faster than sorting, matching experimental data.

Explicit time scales for learning phenomena are derived.

Abstract

We study a continuum model for stochastic reinforcement learning in repeated market entry games. Starting from a discrete-time microscopic learning rule, we derive a Fokker--Planck-type equation for the distribution of agents' propensities and, using a kinetic closure, obtain a nonlinear one-particle equation of a mean-field type. For the resulting Cauchy problem, we prove existence and uniqueness of solutions and analyze their long-time behavior. The PDE captures two key phenomena observed in market entry dynamics: aggregate learning (the average number of entrants approaches market capacity) and sorting (propensities concentrate near extreme behaviors). The model also yields explicit characteristic time scales, showing that aggregate learning occurs faster than sorting, in agreement with experimental and computational evidence.