Empirical Equilibria in Agent-based Economic systems with Learning agents

TL;DR

This paper introduces an agent-based economic simulation environment where agents learn via reinforcement learning, and employs the PSRO algorithm to approximate Nash equilibria, demonstrating improved strategy stability over traditional methods.

Contribution

It combines multi-agent reinforcement learning with the PSRO algorithm in an economic simulation, providing a novel approach to approximating economic equilibria.

Findings

PSRO strategies achieve lower regret than independent MARL.

The environment models heterogeneous agents with distinct objectives.

Linking AI and economic theory for equilibrium analysis.

Abstract

We present an agent-based simulator for economic systems with heterogeneous households, firms, central bank, and government agents. These agents interact to define production, consumption, and monetary flow. Each agent type has distinct objectives, such as households seeking utility from consumption and the central bank targeting inflation and production. We define this multi-agent economic system using an OpenAI Gym-style environment, enabling agents to optimize their objectives through reinforcement learning. Standard multi-agent reinforcement learning (MARL) schemes, like independent learning, enable agents to learn concurrently but do not address whether the resulting strategies are at equilibrium. This study integrates the Policy Space Response Oracle (PSRO) algorithm, which has shown superior performance over independent MARL in games with homogeneous agents, with economic agent-based modeling. We use PSRO to develop agent policies approximating Nash equilibria of the empirical economic game, thereby linking to economic equilibria. Our results demonstrate that PSRO strategies achieve lower regret values than independent MARL strategies in our economic system with four agent types. This work aims to bridge artificial intelligence, economics, and empirical game theory towards future research.