Efficiently Solving Turn-Taking Stochastic Games with Extensive-Form Correlation

TL;DR

This paper introduces polynomial-time algorithms for computing equilibrium strategies in two-player turn-taking stochastic games with extensive-form correlation, advancing the efficiency and applicability of equilibrium computation methods.

Contribution

It presents the first polynomial-time algorithm for SEFCE with commitment in general stochastic games and an efficient approximation algorithm for EFCE that handles stochasticity, approximation, and succinct game representations.

Findings

First polynomial-time algorithm for SEFCE with commitment in stochastic games.

Efficient approximation algorithm for EFCE with optimality and low error dependency.

Algorithms work in succinct graph form, unlike previous tree-based methods.

Abstract

We study equilibrium computation with extensive-form correlation in two-player turn-taking stochastic games. Our main results are two-fold: (1) We give an algorithm for computing a Stackelberg extensive-form correlated equilibrium (SEFCE), which runs in time polynomial in the size of the game, as well as the number of bits required to encode each input number. (2) We give an efficient algorithm for approximately computing an optimal extensive-form correlated equilibrium (EFCE) up to machine precision, i.e., the algorithm achieves approximation error $\varepsilon$ in time polynomial in the size of the game, as well as $\log(1 / \varepsilon)$. Our algorithm for SEFCE is the first polynomial-time algorithm for equilibrium computation with commitment in such a general class of stochastic games. Existing algorithms for SEFCE typically make stronger assumptions such as no chance moves, and are designed for extensive-form games in the less succinct tree form. Our algorithm for approximately optimal EFCE is, to our knowledge, the first algorithm that achieves 3 desiderata simultaneously: approximate optimality, polylogarithmic dependency on the approximation error, and compatibility with stochastic games in the more succinct graph form. Existing algorithms achieve at most 2 of these desiderata, often also relying on additional technical assumptions.