First-Order Softmax Weighted Switching Gradient Method for Distributed Stochastic Minimax Optimization with Stochastic Constraints

TL;DR

This paper introduces a novel first-order Softmax-Weighted Switching Gradient method for distributed stochastic minimax optimization with stochastic constraints, achieving optimal convergence rates and robustness in federated learning scenarios.

Contribution

The paper proposes a new primal-only switching gradient algorithm for distributed minimax problems with stochastic constraints, improving convergence guarantees and stability over existing methods.

Findings

Achieves $ ext{O}(rac{1}{ ext{epsilon}^4})$ oracle complexity under full participation.

Provides a tighter lower bound for the softmax hyperparameter.

Demonstrates effectiveness on Neyman-Pearson and fair classification tasks.

Abstract

This paper addresses the distributed stochastic minimax optimization problem subject to stochastic constraints. We propose a novel first-order Softmax-Weighted Switching Gradient method tailored for federated learning. Under full client participation, our algorithm achieves the standard $\mathcal{O}(\epsilon^{-4})$ oracle complexity to satisfy a unified bound $\epsilon$ for both the optimality gap and feasibility tolerance. We extend our theoretical analysis to the practical partial participation regime by quantifying client sampling noise through a stochastic superiority assumption. Furthermore, by relaxing standard boundedness assumptions on the objective functions, we establish a strictly tighter lower bound for the softmax hyperparameter. We provide a unified error decomposition and establish a sharp $\mathcal{O}(\log\frac{1}{\delta})$ high-probability convergence guarantee. Ultimately, our framework demonstrates that a single-loop primal-only switching mechanism provides a stable alternative for optimizing worst-case client performance, effectively bypassing the hyperparameter sensitivity and convergence oscillations often encountered in traditional primal-dual or penalty-based approaches. We verify the efficacy of our algorithm via experiment on the Neyman-Pearson (NP) classification and fair classification tasks.