Extrinsic Bayesian Optimizations on Manifolds

TL;DR

This paper introduces an extrinsic Bayesian optimization framework for manifolds, leveraging embeddings into Euclidean space to construct Gaussian process kernels, enabling efficient optimization on complex manifolds.

Contribution

The paper presents a novel extrinsic Bayesian optimization method that uses equivariant embeddings to handle Gaussian processes on general manifolds, addressing kernel construction challenges.

Findings

Effective optimization on spheres, Grassmannians, and positive definite matrices.

Scalable algorithms demonstrated through simulations and real data.

Improved performance over existing manifold optimization methods.

Abstract

We propose an extrinsic Bayesian optimization (eBO) framework for general optimization problems on manifolds. Bayesian optimization algorithms build a surrogate of the objective function by employing Gaussian processes and quantify the uncertainty in that surrogate by deriving an acquisition function. This acquisition function represents the probability of improvement based on the kernel of the Gaussian process, which guides the search in the optimization process. The critical challenge for designing Bayesian optimization algorithms on manifolds lies in the difficulty of constructing valid covariance kernels for Gaussian processes on general manifolds. Our approach is to employ extrinsic Gaussian processes by first embedding the manifold onto some higher dimensional Euclidean space via equivariant embeddings and then constructing a valid covariance kernel on the image manifold after the embedding. This leads to efficient and scalable algorithms for optimization over complex manifolds. Simulation study and real data analysis are carried out to demonstrate the utilities of our eBO framework by applying the eBO to various optimization problems over manifolds such as the sphere, the Grassmannian, and the manifold of positive definite matrices.