Cycles in adversarial regularized learning

TL;DR

This paper investigates the dynamic behavior of regularized learning algorithms in zero-sum games, revealing that their trajectories are recurrent and exhibit cycling behavior under various conditions.

Contribution

It demonstrates that regularized learning dynamics in zero-sum games are Poincaré recurrent, showing persistent cycling behavior across different regularizers and game structures.

Findings

Trajectories revisit neighborhoods of initial points infinitely often.

Cycling behavior is robust to different regularizers and utility transformations.

Persistence of cycling in networked zero-sum polymatrix games.

Abstract

Regularized learning is a fundamental technique in online optimization, machine learning and many other fields of computer science. A natural question that arises in these settings is how regularized learning algorithms behave when faced against each other. We study a natural formulation of this problem by coupling regularized learning dynamics in zero-sum games. We show that the system's behavior is Poincar\'e recurrent, implying that almost every trajectory revisits any (arbitrarily small) neighborhood of its starting point infinitely often. This cycling behavior is robust to the agents' choice of regularization mechanism (each agent could be using a different regularizer), to positive-affine transformations of the agents' utilities, and it also persists in the case of networked competition, i.e., for zero-sum polymatrix games.