Stochastic Non-Smooth Non-Convex Optimization with Decision-Dependent Distributions

TL;DR

This paper analyzes stochastic zeroth-order optimization with decision-dependent sampling, providing convergence guarantees and complexity bounds for non-smooth, smooth, and Hessian-Lipschitz non-convex functions.

Contribution

It establishes explicit convergence guarantees and complexity bounds for decision-dependent zeroth-order optimization in non-smooth and smooth settings, improving existing results.

Findings

Convergence guarantee for non-smooth non-convex problems with complexity $ ilde{O}(d^2 ext{delta}^{-3} ext{epsilon}^{-3})$.

Single feedback per iteration suffices to achieve the complexity bounds.

Improved complexity dependence on $ ext{epsilon}$ for Hessian-Lipschitz objectives by a factor of $ ext{epsilon}^{-1/2}$.

Abstract

We study stochastic zeroth-order optimization with decision-dependent distributions, where the sampling law depends on the current decision and only noisy function values are available. For the non-smooth non-convex setting, we establish an explicit convergence guarantee for finding a $(\delta,\epsilon)$-Goldstein stationary point with stochastic zeroth-order oracle (SZO) complexity of $\mathcal{O}(d^2\delta^{-3}\epsilon^{-3})$. In addition, we show that the above complexity can be achieved with single SZO feedback per iteration. We further extend the analysis to smooth and Hessian-Lipschitz objectives, obtaining complexities $\mathcal{O}(d^2\epsilon^{-6})$ and $\mathcal{O}(d^2\epsilon^{-9/2})$, respectively. In the Hessian-Lipschitz case, this improves the best-known dependence on $\epsilon$ for decision-dependent zeroth-order methods by a factor of $\epsilon^{-1/2}$.