Rapid Learning in Constrained Minimax Games with Negative Momentum

TL;DR

This paper introduces a negative momentum technique for constrained minimax games, providing a new framework that improves convergence and performance in both normal and extensive form games, backed by theoretical guarantees and experiments.

Contribution

The paper extends negative momentum methods to constrained minimax games with a novel update framework and provides convergence guarantees, enhancing classic algorithms.

Findings

Significant performance improvements over baselines.

Effective in both normal and extensive form games.

Theoretical convergence guarantees established.

Abstract

In this paper, we delve into the utilization of the negative momentum technique in constrained minimax games. From an intuitive mechanical standpoint, we introduce a novel framework for momentum buffer updating, which extends the findings of negative momentum from the unconstrained setting to the constrained setting and provides a universal enhancement to the classic game-solver algorithms. Additionally, we provide theoretical guarantee of convergence for our momentum-augmented algorithms with entropy regularizer. We then extend these algorithms to their extensive-form counterparts. Experimental results on both Normal Form Games (NFGs) and Extensive Form Games (EFGs) demonstrate that our momentum techniques can significantly improve algorithm performance, surpassing both their original versions and the SOTA baselines by a large margin.