On Zeroth-Order Stochastic Convex Optimization via Random Walks

TL;DR

This paper introduces a zeroth-order stochastic convex optimization method using a random walk on the epigraph, achieving competitive suboptimality rates and robustness to noise without gradient estimation.

Contribution

It presents a novel random walk-based approach for zeroth-order convex optimization that reduces sensitivity to noise and avoids gradient estimation.

Findings

Achieves suboptimality rate of (n^{7}T^{-1/2})

Less sensitive to noisy function evaluations

Uses a random walk (Ball Walk) on the epigraph of the function

Abstract

We propose a method for zeroth order stochastic convex optimization that attains the suboptimality rate of $\tilde{\mathcal{O}}(n^{7}T^{-1/2})$ after $T$ queries for a convex bounded function $f:{\mathbb R}^n\to{\mathbb R}$. The method is based on a random walk (the \emph{Ball Walk}) on the epigraph of the function. The randomized approach circumvents the problem of gradient estimation, and appears to be less sensitive to noisy function evaluations compared to noiseless zeroth order methods.