Balancing Adaptability and Non-exploitability in Repeated Games

TL;DR

This paper introduces LAFF, an expert algorithm for repeated games that balances low regret against various opponent classes while ensuring non-exploitability, a novel combination in multi-agent learning.

Contribution

The paper presents LAFF, the first algorithm to guarantee both low regret and non-exploitability across multiple opponent classes in repeated games.

Findings

LAFF achieves sublinear regret against non-exploitative opponents.

LAFF guarantees linear regret for exploitative opponents.

This work is the first to combine regret guarantees with non-exploitability in multi-agent settings.

Abstract

We study the problem of guaranteeing low regret in repeated games against an opponent with unknown membership in one of several classes. We add the constraint that our algorithm is non-exploitable, in that the opponent lacks an incentive to use an algorithm against which we cannot achieve rewards exceeding some "fair" value. Our solution is an expert algorithm (LAFF) that searches within a set of sub-algorithms that are optimal for each opponent class and uses a punishment policy upon detecting evidence of exploitation by the opponent. With benchmarks that depend on the opponent class, we show that LAFF has sublinear regret uniformly over the possible opponents, except exploitative ones, for which we guarantee that the opponent has linear regret. To our knowledge, this work is the first to provide guarantees for both regret and non-exploitability in multi-agent learning.