Analyzing Micro-Founded General Equilibrium Models with Many Agents using Deep Reinforcement Learning

TL;DR

This paper introduces a deep multi-agent reinforcement learning framework to efficiently find approximate equilibria in complex, microfounded dynamic general equilibrium models with many heterogeneous agents, overcoming previous computational challenges.

Contribution

The authors develop a structured MARL approach with curricula that achieves stable convergence to meaningful solutions in microfounded DGE models, without relying on unrealistic assumptions.

Findings

Successfully finds $$-meta-equilibria in large-scale RBC models.

Demonstrates alignment of learned solutions with economic intuition.

Shows the approach can learn diverse equilibria in open RBC models.

Abstract

Real economies can be modeled as a sequential imperfect-information game with many heterogeneous agents, such as consumers, firms, and governments. Dynamic general equilibrium (DGE) models are often used for macroeconomic analysis in this setting. However, finding general equilibria is challenging using existing theoretical or computational methods, especially when using microfoundations to model individual agents. Here, we show how to use deep multi-agent reinforcement learning (MARL) to find $\epsilon$-meta-equilibria over agent types in microfounded DGE models. Whereas standard MARL fails to learn non-trivial solutions, our structured learning curricula enable stable convergence to meaningful solutions. Conceptually, our approach is more flexible and does not need unrealistic assumptions, e.g., continuous market clearing, that are commonly used for analytical tractability. Furthermore, our end-to-end GPU implementation enables fast real-time convergence with a large number of RL economic agents. We showcase our approach in open and closed real-business-cycle (RBC) models with 100 worker-consumers, 10 firms, and a social planner who taxes and redistributes. We validate the learned solutions are $\epsilon$-meta-equilibria through best-response analyses, show that they align with economic intuitions, and show our approach can learn a spectrum of qualitatively distinct $\epsilon$-meta-equilibria in open RBC models. As such, we show that hardware-accelerated MARL is a promising framework for modeling the complexity of economies based on microfoundations.