Learning to Guide Random Search

TL;DR

This paper introduces a novel online learning method for derivative-free optimization that leverages the low-dimensional manifold structure of high-dimensional functions, significantly reducing sample complexity.

Contribution

The paper proposes a joint learning and optimization approach that models the low-dimensional manifold, improving efficiency over existing derivative-free methods.

Findings

Achieves lower sample complexity than existing algorithms.

Effective on high-dimensional continuous control benchmarks.

Outperforms Augmented Random Search, Bayesian optimization, and CMA-ES.

Abstract

We are interested in derivative-free optimization of high-dimensional functions. The sample complexity of existing methods is high and depends on problem dimensionality, unlike the dimensionality-independent rates of first-order methods. The recent success of deep learning suggests that many datasets lie on low-dimensional manifolds that can be represented by deep nonlinear models. We therefore consider derivative-free optimization of a high-dimensional function that lies on a latent low-dimensional manifold. We develop an online learning approach that learns this manifold while performing the optimization. In other words, we jointly learn the manifold and optimize the function. Our analysis suggests that the presented method significantly reduces sample complexity. We empirically evaluate the method on continuous optimization benchmarks and high-dimensional continuous control problems. Our method achieves significantly lower sample complexity than Augmented Random Search, Bayesian optimization, covariance matrix adaptation (CMA-ES), and other derivative-free optimization algorithms.