Compressible Spectral Mixture Kernels with Sparse Dependency Structures for Gaussian Processes

TL;DR

This paper enhances spectral mixture kernels for Gaussian processes by introducing a novel dependency structure with time and phase modulation, improving model expressiveness and generalization through compression and sparsity techniques.

Contribution

It proposes a new spectral mixture kernel with a dependency structure using cross-convolution and time-phase delays, along with a structure adaptation algorithm for better hyperparameter initialization.

Findings

SMD kernel outperforms traditional SM kernels on synthetic data.

The proposed method improves generalization in real-world applications.

Sparse dependency structures reduce computational complexity.

Abstract

Spectral mixture (SM) kernels comprise a powerful class of generalized kernels for Gaussian processes (GPs) to describe complex patterns. This paper introduces model compression and time- and phase (TP) modulated dependency structures to the original (SM) kernel for improved generalization of GPs. Specifically, by adopting Bienaym\'es identity, we generalize the dependency structure through cross-covariance between the SM components. Then, we propose a novel SM kernel with a dependency structure (SMD) by using cross-convolution between the SM components. Furthermore, we ameliorate the expressiveness of the dependency structure by parameterizing it with time and phase delays. The dependency structure has clear interpretations in terms of spectral density, covariance behavior, and sampling path. To enrich the SMD with effective hyperparameter initialization, compressible SM kernel components, and sparse dependency structures, we introduce a novel structure adaptation (SA) algorithm in the end. A thorough comparative analysis of the SMD on both synthetic and real-life applications corroborates its efficacy.