Learning Local Stackelberg Equilibria from Repeated Interactions with a Learning Agent

TL;DR

This paper introduces a polynomial-time approximation scheme for computing local Stackelberg equilibria in repeated interactions with learning agents, addressing computational intractability of global solutions.

Contribution

It presents the first PTAS for local Stackelberg equilibria in repeated games with learning agents, with proven runtime dependencies.

Findings

The algorithm runs in polynomial time in the size of the agent's action space.

Runtime is exponential in 1/epsilon, which is proven to be unavoidable.

The approach applies within the smoothed analysis framework.

Abstract

Motivated by the question of how a principal can maximize its utility in repeated interactions with a learning agent, we study repeated games between an principal and an agent employing a mean-based learning algorithm. Prior work has shown that computing or even approximating the global Stackelberg value in similar settings can require an exponential number of rounds in the size of the agent's action space, making it computationally intractable. In contrast, we shift focus to the computation of local Stackelberg equilibria and introduce an algorithm that, within the smoothed analysis framework, constitutes a Polynomial Time Approximation Scheme (PTAS) for finding an epsilon-approximate local Stackelberg equilibrium. Notably, the algorithm's runtime is polynomial in the size of the agent's action space yet exponential in (1/epsilon) - a dependency we prove to be unavoidable.