Multiplicative Weights Update with Constant Step-Size in Congestion Games: Convergence, Limit Cycles and Chaos

TL;DR

This paper proves that the Multiplicative Weights Update algorithm converges to Nash equilibria in congestion games with arbitrary learning rates, but certain variants can lead to limit cycles or chaos, highlighting nuanced dynamics.

Contribution

It establishes convergence of MWU with constant step-size in congestion games and reveals potential for complex behaviors in related variants.

Findings

MWU converges to Nash equilibria with arbitrary learning rates.

Certain MWU variants can cause limit cycles or chaos.

A novel connection between MWU and the Baum-Welch algorithm is demonstrated.

Abstract

The Multiplicative Weights Update (MWU) method is a ubiquitous meta-algorithm that works as follows: A distribution is maintained on a certain set, and at each step the probability assigned to element $\gamma$ is multiplied by $(1 -\epsilon C(\gamma))>0$ where $C(\gamma)$ is the "cost" of element $\gamma$ and then rescaled to ensure that the new values form a distribution. We analyze MWU in congestion games where agents use \textit{arbitrary admissible constants} as learning rates $\epsilon$ and prove convergence to \textit{exact Nash equilibria}. Our proof leverages a novel connection between MWU and the Baum-Welch algorithm, the standard instantiation of the Expectation-Maximization (EM) algorithm for hidden Markov models (HMM). Interestingly, this convergence result does not carry over to the nearly homologous MWU variant where at each step the probability assigned to element $\gamma$ is multiplied by $(1 -\epsilon)^{C(\gamma)}$ even for the most innocuous case of two-agent, two-strategy load balancing games, where such dynamics can provably lead to limit cycles or even chaotic behavior.