On the Global Convergence of Stochastic Fictitious Play in Stochastic Games with Turn-based Controllers
Muhammed O. Sayin

TL;DR
This paper introduces a learning dynamic that guarantees almost sure convergence in stochastic games with turn-based controllers, accommodating diverse stage-payoff structures, by combining stochastic fictitious play with value iteration.
Contribution
It presents a novel convergence guarantee for stochastic fictitious play in stochastic games with turn-based controllers, even with varying stage-payoff structures.
Findings
Guarantees almost sure convergence in specified stochastic games.
Handles diverse stage-payoff structures across states.
Integrates stochastic fictitious play with value iteration effectively.
Abstract
This paper presents a learning dynamic with almost sure convergence guarantee for any stochastic game with turn-based controllers (on state transitions) as long as stage-payoffs induce a zero-sum or identical-interest game. Stage-payoffs for different states can even have different structures, e.g., by summing to zero in some states and being identical in others. The dynamics presented combines the classical stochastic fictitious play with value iteration for stochastic games. There are two key properties: (i) players play finite horizon stochastic games with increasing lengths within the underlying infinite-horizon stochastic game, and (ii) the turn-based controllers ensure that the auxiliary stage-games (induced from the continuation payoff estimated) are strategically equivalent to zero-sum or identical-interest games.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEconomic theories and models · Game Theory and Applications · Experimental Behavioral Economics Studies
