Fictitious Play in Markov Games with Single Controller
Muhammed O. Sayin, Kaiqing Zhang, Asuman Ozdaglar

TL;DR
This paper investigates the convergence of fictitious play dynamics to Nash equilibria in Markov games, specifically focusing on a new variant for n-player identical-interest games with a single controller, bridging a gap in game-theoretic analysis.
Contribution
The paper introduces a new fictitious play variant for Markov games and proves its convergence in n-player identical-interest MGs with a single controller, extending existing results.
Findings
Fictitious play converges to NE in certain MGs with a single controller.
Establishes FPP for two-player zero-sum MGs and n-player identical-interest MGs with a single controller.
Bridges the gap between fully competitive and fully cooperative game models.
Abstract
Certain but important classes of strategic-form games, including zero-sum and identical-interest games, have the fictitious-play-property (FPP), i.e., beliefs formed in fictitious play dynamics always converge to a Nash equilibrium (NE) in the repeated play of these games. Such convergence results are seen as a (behavioral) justification for the game-theoretical equilibrium analysis. Markov games (MGs), also known as stochastic games, generalize the repeated play of strategic-form games to dynamic multi-state settings with Markovian state transitions. In particular, MGs are standard models for multi-agent reinforcement learning -- a reviving research area in learning and games, and their game-theoretical equilibrium analyses have also been conducted extensively. However, whether certain classes of MGs have the FPP or not (i.e., whether there is a behavioral justification for equilibrium…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsExperimental Behavioral Economics Studies · Game Theory and Applications · Decision-Making and Behavioral Economics
