An Approximate Subgame-Perfect Equilibrium Computation Technique for Repeated Games

TL;DR

This paper introduces a novel method for approximating the set of subgame-perfect equilibria in discounted repeated games with arbitrary precision, using hypercube partitioning and mathematical programming.

Contribution

It proposes a new algorithm for approximating SPE sets, including strategies extraction, applicable with or without player coordination assumptions.

Findings

The algorithm can approximate SPE sets to any desired accuracy.

Strategies can be represented as finite automata for artificial agents.

Three formulations of the algorithm are provided for different coordination assumptions.

Abstract

This paper presents a technique for approximating, up to any precision, the set of subgame-perfect equilibria (SPE) in discounted repeated games. The process starts with a single hypercube approximation of the set of SPE. Then the initial hypercube is gradually partitioned on to a set of smaller adjacent hypercubes, while those hypercubes that cannot contain any point belonging to the set of SPE are simultaneously withdrawn. Whether a given hypercube can contain an equilibrium point is verified by an appropriate mathematical program. Three different formulations of the algorithm for both approximately computing the set of SPE payoffs and extracting players' strategies are then proposed: the first two that do not assume the presence of an external coordination between players, and the third one that assumes a certain level of coordination during game play for convexifying the set of continuation payoffs after any repeated game history. A special attention is paid to the question of extracting players' strategies and their representability in form of finite automata, an important feature for artificial agent systems.