Sample-efficient Bayesian Optimisation Using Known Invariances

TL;DR

This paper enhances Bayesian optimisation by incorporating known invariances into Gaussian process kernels, significantly improving sample efficiency and enabling successful application to complex real-world problems like nuclear fusion system design.

Contribution

It introduces a novel invariance-aware kernel for Bayesian optimisation, providing theoretical bounds and demonstrating practical improvements over existing methods.

Findings

Invariance-aware kernels outperform vanilla BO in sample efficiency.

Theoretical bounds on information gain and sample complexity are established.

Application to nuclear fusion design shows practical effectiveness.

Abstract

Bayesian optimisation (BO) is a powerful framework for global optimisation of costly functions, using predictions from Gaussian process models (GPs). In this work, we apply BO to functions that exhibit invariance to a known group of transformations. We show that vanilla and constrained BO algorithms are inefficient when optimising such invariant objectives, and provide a method for incorporating group invariances into the kernel of the GP to produce invariance-aware algorithms that achieve significant improvements in sample efficiency. We derive a bound on the maximum information gain of these invariant kernels, and provide novel upper and lower bounds on the number of observations required for invariance-aware BO algorithms to achieve $\epsilon$-optimality. We demonstrate our method's improved performance on a range of synthetic invariant and quasi-invariant functions. We also apply our method in the case where only some of the invariance is incorporated into the kernel, and find that these kernels achieve similar gains in sample efficiency at significantly reduced computational cost. Finally, we use invariant BO to design a current drive system for a nuclear fusion reactor, finding a high-performance solution where non-invariant methods failed.