Structure Discovery in Nonparametric Regression through Compositional Kernel Search

TL;DR

This paper introduces a method for discovering interpretable and effective kernel structures in nonparametric regression by searching over compositional kernels, improving prediction accuracy and interpretability.

Contribution

It proposes a novel search method for compositional kernels that enhances model interpretability and predictive performance in nonparametric regression.

Findings

Outperforms standard kernels on various prediction tasks

Enables interpretable decomposition of functions

Facilitates long-range extrapolation in time-series data

Abstract

Despite its importance, choosing the structural form of the kernel in nonparametric regression remains a black art. We define a space of kernel structures which are built compositionally by adding and multiplying a small number of base kernels. We present a method for searching over this space of structures which mirrors the scientific discovery process. The learned structures can often decompose functions into interpretable components and enable long-range extrapolation on time-series datasets. Our structure search method outperforms many widely used kernels and kernel combination methods on a variety of prediction tasks.