Improved Exploiting Higher Order Smoothness in Derivative-free Optimization and Continuous Bandit

TL;DR

This paper advances the theoretical understanding of derivative-free stochastic convex optimization by improving convergence bounds for higher-order smooth functions with zero-order oracle feedback.

Contribution

It introduces a tighter convergence bound for $eta$-smooth convex optimization, enhancing previous results for higher-order smoothness in derivative-free methods.

Findings

Improved convergence rate bound for $eta$-smooth functions.

Enhanced theoretical guarantees for zero-order stochastic convex optimization.

Better understanding of higher-order smoothness impact on optimization efficiency.

Abstract

We consider $\beta$-smooth (satisfies the generalized Holder condition with parameter $\beta > 2$) stochastic convex optimization problem with zero-order one-point oracle. The best known result was arXiv:2006.07862: $\mathbb{E} \left[f(\overline{x}_N) - f(x^*)\right] = \tilde{\mathcal{O}} \left(\dfrac{n^{2}}{\gamma N^{\frac{\beta-1}{\beta}}} \right)$ in $\gamma$-strongly convex case, where $n$ is the dimension. In this paper we improve this bound: $\mathbb{E} \left[f(\overline{x}_N) - f(x^*)\right] = \tilde{\mathcal{O}} \left(\dfrac{n^{2-\frac{1}{\beta}}}{\gamma N^{\frac{\beta-1}{\beta}}} \right).$