Optimal last-iterate convergence in matrix games with bandit feedback using the log-barrier

TL;DR

This paper demonstrates that using a log-barrier regularization in online mirror descent algorithms achieves optimal last-iterate convergence rates in zero-sum matrix and extensive-form games with high probability.

Contribution

It introduces a novel approach combining log-barrier regularization and dual-focused analysis to attain the theoretical convergence rate Omega(t^{-1/4}) in these game settings.

Findings

Achieves O-tilde(t^{-1/4}) convergence rate with high probability.

Extends the approach to extensive-form games with similar convergence guarantees.

Provides a new method for last-iterate convergence in uncoupled game algorithms.

Abstract

We study the problem of learning minimax policies in zero-sum matrix games. Fiegel et al. (2025) recently showed that achieving last-iterate convergence in this setting is harder when the players are uncoupled, by proving a lower bound on the exploitability gap of Omega(t^{-1/4}). Some online mirror descent algorithms were proposed in the literature for this problem, but none have truly attained this rate yet. We show that the use of a log-barrier regularization, along with a dual-focused analysis, allows this O-tilde(t^{-1/4}) convergence with high-probability. We additionally extend our idea to the setting of extensive-form games, proving a bound with the same rate.