Communication-Efficient Decentralized Stochastic Minimax Optimization

TL;DR

This paper introduces a communication-efficient decentralized stochastic minimax optimization algorithm that reduces local updates and improves communication and sample efficiency, validated on real datasets.

Contribution

It develops a novel accelerated momentum-based decentralized minimax method that outperforms existing algorithms in communication and sample efficiency.

Findings

Reduces local updates to old while maintaining optimal communication complexity.

Achieves superior communication efficiency in experiments on real-world datasets.

Enhances sample complexity compared to previous local gradient descent-ascent methods.

Abstract

In this work, we study decentralized stochastic nonconvex Polyak--{\L}ojasiewicz minimax problems and propose a communication-efficient algorithm. Motivated by the efficiency of local SGD in federated learning, we investigate decentralized minimax algorithms that perform multiple local updates between gossip rounds to improve communication efficiency. Existing results show that the local decentralized gradient descent-ascent algorithm requires an excessive number of local updates, on the order of $\tilde{\mathcal{O}}(\kappa^2\varepsilon^{-2})$ per communication round, to achieve the communication complexity $\tilde{\mathcal{O}}(\kappa^3\varepsilon^{-2})$, where $\varepsilon$ denotes the target accuracy and $\kappa>1$ is the condition number. However, such a large number of local updates can be impractical: it can underexploit available communication resources and exacerbate local drift, as agents may move toward their own local optima. To address this limitation, we develop a local decentralized minimax method that integrates accelerated momentum with local updates. Our method reduces the number of local updates to $\tilde{\mathcal{O}}(\kappa\varepsilon^{-1})$ per communication round while achieving the best-known communication complexity $\mathcal{O}(\kappa^2\varepsilon^{-2})$. Compared with the local gradient descent-ascent method, the proposed method also achieves an enhanced sample complexity. Experiments on robust logistic regression with real-world datasets demonstrate that our method achieves superior communication efficiency over several existing baselines.