Can a One-Point Feedback Zeroth-order Algorithm Achieve Linear Dimension Dependent Sample Complexity?

TL;DR

This paper demonstrates that one-point feedback zeroth-order optimization algorithms can achieve linear dimension-dependent sample complexity, closing the gap with two-point methods and improving efficiency in derivative-free optimization.

Contribution

It proves that one-point feedback zeroth-order algorithms can match the linear dimension dependence of two-point methods, a significant theoretical advancement.

Findings

Achieves linear dimension dependence in sample complexity.

Bridges the gap between one-point and two-point zeroth-order methods.

Provides theoretical proof of improved sample efficiency.

Abstract

We revisit the one-point feedback zeroth-order (ZO) optimization problem, a classical setting in derivative-free optimization where only a single noisy function evaluation is available per query. Compared to their two-point counterparts, existing one-point feedback ZO algorithms typically suffer from poor dimension dependence in their sample complexities -- often quadratic or worse -- even for convex problems. This gap has led to the open question of whether one-point feedback ZO algorithms can match the optimal \emph{linear} dimension dependence achieved by two-point methods. In this work, we answer this question \emph{affirmatively}.