Stochastic gradient-free descents

TL;DR

This paper introduces stochastic gradient-free and accelerated methods with momentum for stochastic optimization, analyzing their convergence behavior and demonstrating their effectiveness in both convex and nonconvex settings.

Contribution

It proposes novel stochastic gradient-free algorithms with momentum, providing theoretical convergence analysis and showing they achieve optimal rates under various conditions.

Findings

Gradient-free methods maintain sublinear convergence with decaying stepsize.

Accelerated methods with momentum achieve faster convergence rates.

All methods converge to stationary points in nonconvex scenarios.

Abstract

In this paper we propose stochastic gradient-free methods and accelerated methods with momentum for solving stochastic optimization problems. All these methods rely on stochastic directions rather than stochastic gradients. We analyze the convergence behavior of these methods under the mean-variance framework, and also provide a theoretical analysis about the inclusion of momentum in stochastic settings which reveals that the momentum term we used adds a deviation of order $\mathcal{O}(1/k)$ but controls the variance at the order $\mathcal{O}(1/k)$ for the $k$th iteration. So it is shown that, when employing a decaying stepsize $\alpha_k=\mathcal{O}(1/k)$, the stochastic gradient-free methods can still maintain the sublinear convergence rate $\mathcal{O}(1/k)$ and the accelerated methods with momentum can achieve a convergence rate $\mathcal{O}(1/k^2)$ in probability for the strongly convex objectives with Lipschitz gradients; and all these methods converge to a solution with a zero expected gradient norm when the objective function is nonconvex, twice differentiable and bounded below.