Natural Evolutionary Search meets Probabilistic Numerics

TL;DR

This paper introduces ProbNES, a new class of algorithms combining Natural Evolution Strategies with Bayesian quadrature, significantly improving sample efficiency in black-box optimization tasks across various applications.

Contribution

The paper presents ProbNES, integrating Bayesian quadrature into NES to enhance sample efficiency and outperform existing methods like Bayesian Optimization in diverse tasks.

Findings

ProbNES outperforms non-probabilistic NES algorithms.

ProbNES surpasses Bayesian Optimization in efficiency.

ProbNES is effective across benchmark, data-driven, and locomotion tasks.

Abstract

Zeroth-order local optimisation algorithms are essential for solving real-valued black-box optimisation problems. Among these, Natural Evolution Strategies (NES) represent a prominent class, particularly well-suited for scenarios where prior distributions are available. By optimising the objective function in the space of search distributions, NES algorithms naturally integrate prior knowledge during initialisation, making them effective in settings such as semi-supervised learning and user-prior belief frameworks. However, due to their reliance on random sampling and Monte Carlo estimates, NES algorithms can suffer from limited sample efficiency. In this paper, we introduce a novel class of algorithms, termed Probabilistic Natural Evolutionary Strategy Algorithms (ProbNES), which enhance the NES framework with Bayesian quadrature. We show that ProbNES algorithms consistently outperforms their non-probabilistic counterparts as well as global sample efficient methods such as Bayesian Optimisation (BO) or $\pi$BO across a wide range of tasks, including benchmark test functions, data-driven optimisation tasks, user-informed hyperparameter tuning tasks and locomotion tasks.