The complexity of solving reachability games using value and strategy iteration

TL;DR

This paper establishes that both value and strategy iteration algorithms for two-player reachability games have doubly-exponential worst-case complexity, with even simple cases requiring exponential iterations for approximation.

Contribution

It provides tight upper and lower bounds on the iteration complexity of value and strategy iteration algorithms for reachability games, revealing their doubly-exponential nature.

Findings

Both algorithms have doubly-exponential worst-case iteration bounds.

Even with a single non-terminal position, exponential lower bounds apply.

The results highlight fundamental computational limitations of these algorithms.

Abstract

Two standard algorithms for approximately solving two-player zero-sum concurrent reachability games are value iteration and strategy iteration. We prove upper and lower bounds of 2^(m^(Theta(N))) on the worst case number of iterations needed by both of these algorithms for providing non-trivial approximations to the value of a game with N non-terminal positions and m actions for each player in each position. In particular, both algorithms have doubly-exponential complexity. Even when the game given as input has only one non-terminal position, we prove an exponential lower bound on the worst case number of iterations needed to provide non-trivial approximations.