Enabling First-Order Gradient-Based Learning for Equilibrium Computation in Markets

TL;DR

This paper introduces a smoothing technique for differentiable market simulations, enabling efficient first-order gradient-based equilibrium computation, which outperforms zeroth-order methods in accuracy and speed.

Contribution

The paper proposes a novel smoothing method that makes non-differentiable market simulations amenable to first-order gradient methods, with theoretical bias bounds and empirical validation.

Findings

The smoothing technique reduces variance in gradient estimates.

First-order methods outperform zeroth-order in accuracy.

The approach improves computational efficiency in equilibrium computation.

Abstract

Understanding and analyzing markets is crucial, yet analytical equilibrium solutions remain largely infeasible. Recent breakthroughs in equilibrium computation rely on zeroth-order policy gradient estimation. These approaches commonly suffer from high variance and are computationally expensive. The use of fully differentiable simulators would enable more efficient gradient estimation. However, the discrete allocation of goods in economic simulations is a non-differentiable operation. This renders the first-order Monte Carlo gradient estimator inapplicable and the learning feedback systematically misleading. We propose a novel smoothing technique that creates a surrogate market game, in which first-order methods can be applied. We provide theoretical bounds on the resulting bias which justifies solving the smoothed game instead. These bounds also allow choosing the smoothing strength a priori such that the resulting estimate has low variance. Furthermore, we validate our approach via numerous empirical experiments. Our method theoretically and empirically outperforms zeroth-order methods in approximation quality and computational efficiency.