Gradient-Free Optimization for Non-Smooth Saddle Point Problems under Adversarial Noise

TL;DR

This paper introduces an optimal zeroth-order method for non-smooth saddle point problems under adversarial noise, improving efficiency and robustness compared to existing algorithms.

Contribution

It presents a simple, optimal zeroth-order stochastic mirror descent algorithm for non-smooth saddle point problems with adversarial noise, including a restart technique for duality gap growth.

Findings

Optimal oracle call complexity achieved

Handles bounded and Lipschitz noise, including adversarial cases

Provides convergence analysis in expectation and probability

Abstract

We consider non-smooth saddle point optimization problems. To solve these problems, we propose a zeroth-order method under bounded or Lipschitz continuous noise, possible adversarial. In contrast to the state-of-the-art algorithms, our algorithm is optimal in terms of both criteria: oracle calls complexity and the maximum value of admissible noise. The proposed method is simple and easy to implement as it is built on zeroth-order version of the stochastic mirror descent. The convergence analysis is given in terms of the average and probability. We also pay special attention to the duality gap $r$-growth condition $(r\geq 1)$, for which we provide a modification of our algorithm using the restart technique. We also comment on infinite noise variance and upper bounds in the case of Lipschitz noise. The results obtained in this paper are significant not only for saddle point problems but also for convex optimization.