The Kernel Manifold: A Geometric Approach to Gaussian Process Model Selection

TL;DR

This paper introduces a geometric Bayesian optimization method for selecting Gaussian Process kernels by embedding kernel space into a continuous manifold, improving model performance and efficiency.

Contribution

It proposes a novel kernel-of-kernels geometric framework using MDS embedding for efficient Gaussian Process kernel selection.

Findings

Achieves superior predictive accuracy on benchmarks and real-world datasets.

Provides a stable and efficient Bayesian optimization landscape for kernel search.

Enhances uncertainty calibration in Gaussian Process models.

Abstract

Gaussian Process (GP) regression is a powerful nonparametric Bayesian framework, but its performance depends critically on the choice of covariance kernel. Selecting an appropriate kernel is therefore central to model quality, yet remains one of the most challenging and computationally expensive steps in probabilistic modeling. We present a Bayesian optimization framework built on kernel-of-kernels geometry, using expected divergence-based distances between GP priors to explore kernel space efficiently. A multidimensional scaling (MDS) embedding of this distance matrix maps a discrete kernel library into a continuous Euclidean manifold, enabling smooth BO. In this formulation, the input space comprises kernel compositions, the objective is the log marginal likelihood, and featurization is given by the MDS coordinates. When the divergence yields a valid metric, the embedding preserves geometry and produces a stable BO landscape. We demonstrate the approach on synthetic benchmarks, real-world time-series datasets, and an additive manufacturing case study predicting melt-pool geometry, achieving superior predictive accuracy and uncertainty calibration relative to baselines including Large Language Model (LLM)-guided search. This framework establishes a reusable probabilistic geometry for kernel search, with direct relevance to GP modeling and deep kernel learning.