Uncoupled and Convergent Learning in Monotone Games under Bandit Feedback

TL;DR

This paper introduces a mirror-descent-based algorithm for monotone games under bandit feedback, achieving convergence and no-regret properties, with improved rates in strongly monotone cases and extensions to time-varying scenarios.

Contribution

It presents the first non-asymptotic convergence results for monotone games with bandit feedback and develops an algorithm with improved convergence rates for strongly monotone games.

Findings

Converges in $O(T^{-1/4})$ under bandit feedback.

Achieves $O(T^{-1/2})$ convergence in strongly monotone games.

Extends to time-varying monotone games with improved equilibrium tracking.

Abstract

We study the problem of no-regret learning algorithms for general monotone and smooth games and their last-iterate convergence properties. Specifically, we investigate the problem under bandit feedback and strongly uncoupled dynamics, which allows modular development of the multi-player system that applies to a wide range of real applications. We propose a mirror-descent-based algorithm, which converges in $O(T^{-1/4})$ and is also no-regret. The result is achieved by a dedicated use of two regularizations and the analysis of the fixed point thereof. The convergence rate is further improved to $O(T^{-1/2})$ in the case of strongly monotone games. Motivated by practical tasks where the game evolves over time, the algorithm is extended to time-varying monotone games. We provide the first non-asymptotic result in converging monotone games and give improved results for equilibrium tracking games.