Anytime-Constrained Equilibria in Polynomial Time

TL;DR

This paper introduces anytime-constrained equilibria in Markov games, providing computational methods including fixed-parameter tractable and polynomial-time algorithms, with optimal approximation guarantees under complexity assumptions.

Contribution

It extends anytime constraints to Markov games, develops a comprehensive theory, and offers new algorithms for computing equilibria with optimal approximation guarantees.

Findings

Fixed-parameter tractable algorithm for ACE

Polynomial-time approximate algorithm for ACE

Optimal approximation guarantees assuming P ≠ NP

Abstract

We extend anytime constraints to the Markov game setting and the corresponding solution concept of an anytime-constrained equilibrium (ACE). Then, we present a comprehensive theory of anytime-constrained equilibria that includes (1) a computational characterization of feasible policies, (2) a fixed-parameter tractable algorithm for computing ACE, and (3) a polynomial-time algorithm for approximately computing ACE. Since computing a feasible policy is NP-hard even for two-player zero-sum games, our approximation guarantees are optimal so long as $P \neq NP$. We also develop the first theory of efficient computation for action-constrained Markov games, which may be of independent interest.