Multigrid methods for two-player zero-sum stochastic games

TL;DR

This paper introduces a fast multigrid-based numerical algorithm for large-scale two-player zero-sum stochastic games, improving computational efficiency through a multi-level policy iteration approach.

Contribution

It combines policy iteration with algebraic multigrid methods to efficiently solve large-scale stochastic games and Isaacs equations, including a multi-level policy iteration scheme.

Findings

Significant reduction in computation time for large-scale problems

Effective application to discretized Isaacs equations and variational inequalities

Demonstrated efficiency improvements with numerical tests

Abstract

We present a fast numerical algorithm for large scale zero-sum stochastic games with perfect information, which combines policy iteration and algebraic multigrid methods. This algorithm can be applied either to a true finite state space zero-sum two player game or to the discretization of an Isaacs equation. We present numerical tests on discretizations of Isaacs equations or variational inequalities. We also present a full multi-level policy iteration, similar to FMG, which allows to improve substantially the computation time for solving some variational inequalities.