Zeroth-Order Non-smooth Non-convex Optimization via Gaussian Smoothing

TL;DR

This paper introduces a novel zeroth-order stochastic subgradient method using Gaussian smoothing for nonsmooth, nonconvex optimization, achieving almost sure convergence to Clarke stationary points with noisy function evaluations.

Contribution

It extends gradient-free optimization techniques to nonsmooth nonconvex problems by establishing bounded error approximations and designing a coupled two-timescale algorithm with convergence guarantees.

Findings

First zeroth-order method with almost sure convergence for constrained nonsmooth nonconvex problems.

Gaussian smoothing provides bounded Clarke subdifferential approximation.

Algorithm converges to a neighborhood of Clarke stationary points almost surely.

Abstract

This paper addresses stochastic optimization of Lipschitz-continuous, nonsmooth and nonconvex objectives over compact convex sets, where only noisy function evaluations are available. While gradient-free methods have been developed for smooth nonconvex problems, extending these techniques to the nonsmooth setting remains challenging. The primary difficulty arises from the absence of a Taylor series expansion for Clarke subdifferentials, which limits the ability to approximate and analyze the behavior of the objective function in a neighborhood of a point. We propose a two time-scale zeroth-order projected stochastic subgradient method leveraging Gaussian smoothing to approximate Clarke subdifferentials. First, we establish that the expectation of the Gaussian-smoothed subgradient lies within an explicitly bounded error of the Clarke subdifferential, a result that extends prior analyses beyond convex/smooth settings. Second, we design a novel algorithm with coupled updates: a fast timescale tracks the subgradient approximation, while a slow timescale drives convergence. Using continuous-time dynamical systems theory and robust perturbation analysis, we prove that iterates converge almost surely to a neighborhood of the set of Clarke stationary points, with neighborhood size controlled by the smoothing parameter. To our knowledge, this is the first zeroth-order method achieving almost sure convergence for constrained nonsmooth nonconvex optimization problems.