Games played by Exponential Weights Algorithms

TL;DR

This paper investigates the convergence behavior of exponential weights algorithms in repeated games, revealing conditions under which players' strategies converge to Nash equilibria or specific equilibrium sets.

Contribution

It provides new theoretical insights into the last-iterate convergence of exponential weights algorithms in game-theoretic settings with constant learning rates.

Findings

Probability of playing strict Nash equilibria converges to 0 or 1.

Limit points belong to Nash Equilibria with Equalizing Payoffs.

In strong coordination games, strategies converge to strict Nash equilibria.

Abstract

This paper studies the last-iterate convergence properties of the exponential weights algorithm with constant learning rates. We consider a repeated interaction in discrete time, where each player uses an exponential weights algorithm characterized by an initial mixed action and a fixed learning rate, so that the mixed action profile $p^t$ played at stage $t$ follows an homogeneous Markov chain. At first, we show that whenever a strict Nash equilibrium exists, the probability to play a strict Nash equilibrium at the next stage converges almost surely to 0 or 1. Secondly, we show that the limit of $p^t$, whenever it exists, belongs to the set of ``Nash Equilibria with Equalizing Payoffs''. Thirdly, we show that in strong coordination games, where the payoff of a player is positive on the diagonal and 0 elsewhere, $p^t$ converges almost surely to one of the strict Nash equilibria. We conclude with open questions.