The AI Economist: Optimal Economic Policy Design via Two-level Deep Reinforcement Learning

TL;DR

The paper introduces the AI Economist, a two-level deep reinforcement learning framework that models economic agents and a social planner to optimize policies like taxation, outperforming traditional methods in complex economic simulations.

Contribution

It presents a novel two-level deep RL approach for economic policy design, capable of learning rational behaviors and optimizing policies in complex, dynamic economies.

Findings

Recovers optimal tax policies in simple economies

Improves social welfare and equality-productivity trade-offs in complex economies

Handles emergent tax-gaming strategies effectively

Abstract

AI and reinforcement learning (RL) have improved many areas, but are not yet widely adopted in economic policy design, mechanism design, or economics at large. At the same time, current economic methodology is limited by a lack of counterfactual data, simplistic behavioral models, and limited opportunities to experiment with policies and evaluate behavioral responses. Here we show that machine-learning-based economic simulation is a powerful policy and mechanism design framework to overcome these limitations. The AI Economist is a two-level, deep RL framework that trains both agents and a social planner who co-adapt, providing a tractable solution to the highly unstable and novel two-level RL challenge. From a simple specification of an economy, we learn rational agent behaviors that adapt to learned planner policies and vice versa. We demonstrate the efficacy of the AI Economist on the problem of optimal taxation. In simple one-step economies, the AI Economist recovers the optimal tax policy of economic theory. In complex, dynamic economies, the AI Economist substantially improves both utilitarian social welfare and the trade-off between equality and productivity over baselines. It does so despite emergent tax-gaming strategies, while accounting for agent interactions and behavioral change more accurately than economic theory. These results demonstrate for the first time that two-level, deep RL can be used for understanding and as a complement to theory for economic design, unlocking a new computational learning-based approach to understanding economic policy.