DAMA: A Unified Accelerated Approach for Decentralized Nonconvex Minimax Optimization-Part II: Convergence and Performance Analyses

TL;DR

This paper provides convergence and performance analyses for DAMA, an accelerated decentralized minimax optimization algorithm, demonstrating its effectiveness and refined bounds for various scenarios in multi-agent networks.

Contribution

It offers the first comprehensive convergence and performance bounds for DAMA, validating its accelerated capabilities in decentralized nonconvex minimax problems.

Findings

Unified performance bounds for DAMA

Refined guarantees for specific algorithm variants

Demonstrated superior performance on sparse networks

Abstract

In Part I of this work [1], we developed an accelerated algorithmic framework, DAMA (Decentralized Accelerated Minimax Approach), for nonconvex Polyak-Lojasiewicz (PL) minimax optimization over decentralized multi-agent networks. To further enhance convergence in online and offline scenarios, Part I of this work [1] also proposed a novel accelerated gradient estimator, namely, GRACE (GRadient ACceleration Estimator), which unifies several momentum-based methods (e.g., STORM) and loopless variance-reduction techniques (e.g., PAGE, Loopless SARAH), thereby enabling accelerated gradient updates within DAMA. Part I reported a unified performance bound for DAMA and refined guarantees for specific algorithmic instances, demonstrating the superior performance of several new variants on sparsely connected networks. In this Part II, we focus on the convergence and performance bounds that substantiate the main results presented in Part I [1]. In particular, we establish a unified performance bound for DAMA using the transformed recursion derived in Part I and subsequently refine this bound for its various special cases.