Gradient Free Minimax Optimization: Variance Reduction and Faster Convergence

TL;DR

This paper introduces a novel zeroth-order variance reduction algorithm for nonconvex-strongly-concave minimax problems, achieving the best known query complexity and outperforming previous methods, with demonstrated effectiveness in black-box robust optimization.

Contribution

It presents the first zeroth-order minimax optimization method with variance reduction that attains optimal complexity bounds without relying on diminishing stepsizes.

Findings

Achieves query complexity of O(κ(d_1 + d_2)ε^{-3})

Outperforms all previous complexity bounds by orders of magnitude

Demonstrates superior performance in black-box distributional robust optimization

Abstract

Many important machine learning applications amount to solving minimax optimization problems, and in many cases there is no access to the gradient information, but only the function values. In this paper, we focus on such a gradient-free setting, and consider the nonconvex-strongly-concave minimax stochastic optimization problem. In the literature, various zeroth-order (i.e., gradient-free) minimax methods have been proposed, but none of them achieve the potentially feasible computational complexity of $\mathcal{O}(\epsilon^{-3})$ suggested by the stochastic nonconvex minimization theorem. In this paper, we adopt the variance reduction technique to design a novel zeroth-order variance reduced gradient descent ascent (ZO-VRGDA) algorithm. We show that the ZO-VRGDA algorithm achieves the best known query complexity of $\mathcal{O}(\kappa(d_1 + d_2)\epsilon^{-3})$, which outperforms all previous complexity bound by orders of magnitude, where $d_1$ and $d_2$ denote the dimensions of the optimization variables and $\kappa$ denotes the condition number. In particular, with a new analysis technique that we develop, our result does not rely on a diminishing or accuracy-dependent stepsize usually required in the existing methods. To our best knowledge, this is the first study of zeroth-order minimax optimization with variance reduction. Experimental results on the black-box distributional robust optimization problem demonstrates the advantageous performance of our new algorithm.