Gaussian Processes and Reproducing Kernels: Connections and Equivalences

TL;DR

This paper explores the fundamental connections and equivalences between Gaussian processes and reproducing kernel Hilbert spaces, providing a unifying perspective that bridges probabilistic and non-probabilistic kernel methods in machine learning and statistics.

Contribution

It establishes a unifying framework linking Gaussian processes and RKHS, clarifying their relationships across various applications in regression, interpolation, and statistical dependence.

Findings

Unified perspective based on Gaussian Hilbert space and RKHS

Connections established for regression, interpolation, and integration

Bridging probabilistic and non-probabilistic kernel methods

Abstract

This monograph studies the relations between two approaches using positive definite kernels: probabilistic methods using Gaussian processes, and non-probabilistic methods using reproducing kernel Hilbert spaces (RKHS). They are widely studied and used in machine learning, statistics, and numerical analysis. Connections and equivalences between them are reviewed for fundamental topics such as regression, interpolation, numerical integration, distributional discrepancies, and statistical dependence, as well as for sample path properties of Gaussian processes. A unifying perspective for these equivalences is established, based on the equivalence between the Gaussian Hilbert space and the RKHS. The monograph serves as a basis to bridge many other methods based on Gaussian processes and reproducing kernels, which are developed in parallel by the two research communities.