Efficient Uncoupled Learning Dynamics with $\tilde{O}\!\left(T^{-1/4}\right)$ Last-Iterate Convergence in Bilinear Saddle-Point Problems over Convex Sets under Bandit Feedback

TL;DR

This paper introduces a computationally efficient uncoupled learning algorithm for bilinear saddle-point problems with bandit feedback, achieving a last-iterate convergence rate of O(T^{-1/4}) in convex settings.

Contribution

The paper presents a novel uncoupled learning algorithm that guarantees last-iterate convergence in bandit feedback scenarios with a O(T^{-1/4}) rate, combining experimental design and FTRL techniques.

Findings

Achieves last-iterate convergence with O(T^{-1/4}) rate.

Requires only an efficient linear optimization oracle.

Applicable to convex action sets with bandit feedback.

Abstract

In this paper, we study last-iterate convergence of learning algorithms in bilinear saddle-point problems, a preferable notion of convergence that captures the day-to-day behavior of learning dynamics. We focus on the challenging setting where players select actions from compact convex sets and receive only bandit feedback. Our main contribution is the design of an uncoupled learning algorithm that guarantees last-iterate convergence to the Nash equilibrium with high probability. We establish a convergence rate of $\tilde{O}(T^{-1/4})$ up to polynomial factors in problem parameters. Crucially, our proposed algorithm is computationally efficient, requiring only an efficient linear optimization oracle over the players' compact action sets. The algorithm is obtained by combining techniques from experimental design and the classic Follow-The-Regularized-Leader (FTRL) framework, with a carefully chosen regularizer function tailored to the geometry of the action set of each learner.