Infinite Horizon Markov Economies
Denizalp Goktas, Sadie Zhao, Yiling Chen, Amy Greenwald
TL;DR
This paper introduces Markov pseudo-games, a new framework generalizing Markov games and pseudo-games, to model infinite-horizon economies, proves equilibrium existence, and develops polynomial-time solution methods with practical neural network implementations.
Contribution
It defines Markov pseudo-games, proves equilibrium existence, introduces a polynomial-time solution method, and applies it to infinite-horizon Markov exchange economies with neural network-based computation.
Findings
Proved existence of equilibrium in Markov pseudo-games.
Developed a polynomial-time convergence solution method.
Successfully computed recursive Radner equilibria using neural networks.
Abstract
In this paper, we study a generalization of Markov games and pseudo-games that we call Markov pseudo-games, which, like the former, captures time and uncertainty, and like the latter, allows for the players' actions to determine the set of actions available to the other players. In the same vein as Arrow and Debreu, we intend for this model to be rich enough to encapsulate a broad mathematical framework for modeling economies. We then prove the existence of a game-theoretic equilibrium in our model, which in turn implies the existence of a general equilibrium in the corresponding economies. Finally, going beyond Arrow and Debreu, we introduce a solution method for Markov pseudo-games and prove its polynomial-time convergence. We then provide an application of Markov pseudo-games to infinite-horizon Markov exchange economies, a stochastic economic model that extends Radner's stochastic…
Peer Reviews
Decision·ICLR 2026 Poster
1. The notion of Markov pseudo-games is conceptually elegant and unifies concepts in game theory and general equilibrium theory in a dynamic setting. The theoretical results are rigorous and intense, and build on a well-established foundation. 2. The mapping from recursive Radner equilibria to GMPE of MPGs appears novel, extending previous equilibrium existence results that were limited to representative-agent or finite-horizon settings. 3. The application of deep multi-agent RL to macroeconomi
1. Experiments are limited to synthetic, small-scale economies. The experimental section focuses on numerical metrics (exploitability, Bellman error) but lacks economic interpretation of the learned equilibria (e.g., consumption paths, asset prices, welfare implications). This limits the insight for economics audiences. 2. Notational issue: $\mathcal{F}(\boldsymbol{\pi})$ is used without giving definition, where $\boldsymbol{\pi}$ is the joint policy profile of $n$-agents. The only definition I
The paper considers a novel and challenging problem. Also, it has a good motivation. The proposed solution approach is also novel. The proposed formulation is particularly interesting, as it unifies infinite-horizon stochastic economies (relevant for macroeconomics area) with incomplete markets using a common theoretical model. The notion of recursive Radner equilibria in this framework is good; it offers an understanding of the market dynamics. It seems that Roy Radner was not awarded a Noble
The paper is primarily theoretical, with limited experimental validation. Thus, we have very limited view of the performance of the proposed approach across networks of varying sizes both in terms of the number of agents and the size of the state space. Hence, the important aspect of scalability and robustness remains underexplored. There are many assumptions that are part of the paper’s analysis (to be expected in a theoretically oriented paper). For example, Assumption 6. But, it is deferred
- The framework the authors propose is novel. It is nice to see that the authors model a real-world economic scenario using their proposed framework. - The paper extends the existential result of RRE in infinite horizon Markov exchange economies. - The paper provides a provable efficient algorithm that converge to a GMPE in Markov pseudo-games and RRE in infinite horizon Markov exchange economies. - Numerical results are provided where the authors model agents as deep learning neural networks
- The notations of this paper is quite dense and difficult to follow. Some notations are used before being properly defined (i.e. $D_{\varphi}(\pi)$ in line 158). - One of the interesting feature of Markov pseudo-games is that each agent's action space depends on actions of other agents. However, Assumption 3 largely simplifies that the problem into a standard min-max optimization problem without any coupling dynamics. This assumption appears overly strong and undermines the main motivation of
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEconomic theories and models