Hierarchical Multiagent Reinforcement Learning for Multi-Group Tax Game

TL;DR

This paper introduces a hierarchical multi-agent reinforcement learning framework with curriculum learning and a closed-loop update mechanism to model and optimize taxation policies in multi-group economic games, addressing stability and convergence challenges.

Contribution

It presents a novel bilevel MARL framework for multi-group taxation games, incorporating curriculum learning and a closed-loop update to enhance training stability and policy effectiveness.

Findings

The method learns stable, sustainable tax policies.

It extends game duration by 60.92% compared to baseline.

It reduces GDP disparities among governments by 44.12%.

Abstract

Reinforcement learning has increasingly been applied to economic decision-making, including taxation, public spending, and labor supply. However, existing RL-based economic models typically consider only a single government-household group, overlooking strategic interactions among competing governments. To address this limitation, we formulate taxation as a hierarchical multi-group game. Within each group, the government and households form a leader--follower game, while governments compete across groups through strategic fiscal policies. This coupled structure is difficult to solve using standard multi-agent reinforcement learning (MARL) methods. We therefore propose a bilevel MARL framework with \textit{Curriculum Learning} and a \textit{Closed-Loop Sequential Update} mechanism to improve training stability and convergence. We instantiate the framework in a taxation simulation environment grounded in classical economic models, supporting the evaluation of taxation policies under inter-group competition. Experiments show that the proposed method learns stable and sustainable tax policies. Compared with a two-group baseline without the proposed mechanisms, our approach avoids premature game collapse, extends the effective game duration by 60.92\%, and reduces GDP disparities among governments by 44.12\%.