Ruppert-Polyak averaging for Stochastic Order Oracle

TL;DR

This paper introduces an improved covariance matrix estimation method for the Stochastic Order Oracle, enhancing convergence rate accuracy in black-box optimization, supported by theoretical analysis and numerical experiments.

Contribution

It presents a novel covariance estimation technique that improves the asymptotic convergence analysis of the Stochastic Order Oracle in black-box optimization.

Findings

Enhanced covariance matrix estimation accuracy

Improved convergence rate predictions

Empirical validation through numerical experiments

Abstract

Black-box optimization, a rapidly growing field, faces challenges due to limited knowledge of the objective function's internal mechanisms. One promising approach to address this is the Stochastic Order Oracle Concept. This concept, similar to other Order Oracle Concepts, relies solely on relative comparisons of function values without requiring access to the exact values. This paper presents a novel, improved estimation of the covariance matrix for the asymptotic convergence of the Stochastic Order Oracle Concept. Our work surpasses existing research in this domain by offering a more accurate estimation of asymptotic convergence rate. Finally, numerical experiments validate our theoretical findings, providing strong empirical support for our proposed approach.