Min-Max Optimization Requires Exponentially Many Queries

TL;DR

This paper proves that finding approximate stationary points in nonconvex-nonconcave min-max optimization requires exponentially many queries in the problem's dimension or accuracy parameter.

Contribution

It establishes a fundamental exponential lower bound on the query complexity for min-max optimization with gradient access.

Findings

Any algorithm needs exponentially many queries in 1/ε or dimension d.

The lower bound applies to algorithms with oracle access to the function and its gradient.

This result highlights inherent difficulty in nonconvex-nonconcave min-max problems.

Abstract

We study the query complexity of min-max optimization of a nonconvex-nonconcave function $f$ over $[0,1]^d \times [0,1]^d$. We show that, given oracle access to $f$ and to its gradient $\nabla f$, any algorithm that finds an $\varepsilon$-approximate stationary point must make a number of queries that is exponential in $1/\varepsilon$ or $d$.