Universal Complexity Bounds Based on Value Iteration for Stochastic Mean Payoff Games and Entropy Games

TL;DR

This paper introduces value iteration algorithms for solving various stochastic mean-payoff and entropy games, providing complexity bounds and polynomial-time solutions for fixed-rank cases.

Contribution

It develops a unified value iteration framework with complexity bounds and applies it to improve solutions for mean-payoff and entropy games with fixed parameters.

Findings

Number of oracle calls is polynomial in dimension and inverse separation.

Turn-based mean-payoff games with fixed random positions are solvable in pseudo-polynomial time.

Entropy games with fixed rank can be solved in polynomial time, extended to weighted cases.

Abstract

We develop value iteration-based algorithms to solve in a unified manner different classes of combinatorial zero-sum games with mean-payoff type rewards. These algorithms rely on an oracle, evaluating the dynamic programming operator up to a given precision. We show that the number of calls to the oracle needed to determine exact optimal (positional) strategies is, up to a factor polynomial in the dimension, of order R/sep, where the "separation" sep is defined as the minimal difference between distinct values arising from strategies, and R is a metric estimate, involving the norm of approximate sub and super-eigenvectors of the dynamic programming operator. We illustrate this method by two applications. The first one is a new proof, leading to improved complexity estimates, of a theorem of Boros, Elbassioni, Gurvich and Makino, showing that turn-based mean-payoff games with a fixed number of random positions can be solved in pseudo-polynomial time. The second one concerns entropy games, a model introduced by Asarin, Cervelle, Degorre, Dima, Horn and Kozyakin. The rank of an entropy game is defined as the maximal rank among all the ambiguity matrices determined by strategies of the two players. We show that entropy games with a fixed rank, in their original formulation, can be solved in polynomial time, and that an extension of entropy games incorporating weights can be solved in pseudo-polynomial time under the same fixed rank condition.