Fast Decentralized Gradient Tracking for Federated Minimax Optimization with Local Updates

TL;DR

This paper introduces -GT-Minimax, a decentralized federated minimax optimization algorithm that combines local updates and gradient tracking, achieving improved communication efficiency and convergence in nonconvex-strongly-concave settings.

Contribution

It proposes a novel decentralized minimax algorithm with local updates and gradient tracking, enhancing convergence and robustness in federated learning.

Findings

Achieves superior convergence rate compared to existing methods.

Effectively handles data heterogeneity in federated settings.

Demonstrates communication efficiency in nonconvex-strongly-concave problems.

Abstract

Federated learning (FL) for minimax optimization has emerged as a powerful paradigm for training models across distributed nodes/clients while preserving data privacy and model robustness on data heterogeneity. In this work, we delve into the decentralized implementation of federated minimax optimization by proposing \texttt{K-GT-Minimax}, a novel decentralized minimax optimization algorithm that combines local updates and gradient tracking techniques. Our analysis showcases the algorithm's communication efficiency and convergence rate for nonconvex-strongly-concave (NC-SC) minimax optimization, demonstrating a superior convergence rate compared to existing methods. \texttt{K-GT-Minimax}'s ability to handle data heterogeneity and ensure robustness underscores its significance in advancing federated learning research and applications.