Upper bounds on the maximum admissible level of noise in zeroth-order optimisation

TL;DR

This paper establishes theoretical upper bounds on the maximum noise level tolerable in zeroth-order convex optimization, extending to strongly convex and smooth cases, and proposes a grid-search algorithm for improved bounds in simplex-constrained problems.

Contribution

It introduces an information-theoretic framework to derive upper bounds on noise levels in zeroth-order optimization and proposes an algorithm that outperforms previous bounds in certain constrained settings.

Findings

Derived upper bounds for strongly convex and smooth problems.

Demonstrated the effectiveness of a grid-search algorithm for simplex constraints.

Provided asymptotic estimates related to problem dimensionality.

Abstract

In this paper, we leverage an information-theoretic upper bound on the maximum admissible level of noise (MALN) in convex Lipschitz-continuous zeroth-order optimisation to establish corresponding upper bounds for classes of strongly convex and smooth problems. We derive these bounds through non-constructive proofs via optimal reductions. Furthermore, we demonstrate that by employing a one-dimensional grid-search algorithm, one can devise an algorithm for simplex-constrained optimisation that offers a superior upper bound on the MALN compared to the case of ball-constrained optimisation and estimates asymptotic in dimensionality.