Efficient Stackelberg Strategies for Finitely Repeated Games

TL;DR

This paper develops efficient algorithms for computing approximate Stackelberg equilibria in finitely repeated, no-discounting games, demonstrating improved leader strategies and analyzing computational hardness in multi-player settings.

Contribution

It introduces two novel algorithms for approximate Stackelberg strategies with different trade-offs and analyzes their convergence rates and computational complexity.

Findings

Algorithms achieve $1/T$ and $1/T^{0.25}$ convergence rates.

Leader strategies outperform single-round Stackelberg in many cases.

Approximating Stackelberg value in three-player games is computationally hard.

Abstract

We study Stackelberg equilibria in finitely repeated games, where the leader commits to a strategy that picks actions in each round and can be adaptive to the history of play (i.e. they commit to an algorithm). In particular, we study static repeated games with no discounting. We give efficient algorithms for finding approximate Stackelberg equilibria in this setting, along with rates of convergence depending on the time horizon $T$. In many cases, these algorithms allow the leader to do much better on average than they can in the single-round Stackelberg. We give two algorithms, one computing strategies with an optimal $\frac{1}{T}$ rate at the expense of an exponential dependence on the number of actions, and another (randomized) approach computing strategies with no dependence on the number of actions but a worse dependence on $T$ of $\frac{1}{T^{0.25}}$. Both algorithms build upon a linear program to produce simple automata leader strategies and induce corresponding automata best-responses for the follower. We complement these results by showing that approximating the Stackelberg value in three-player finite-horizon repeated games is a computationally hard problem via a reduction from balanced vertex cover.