Scalable mixed-domain Gaussian process modeling and model reduction for longitudinal data

TL;DR

This paper introduces a scalable basis function approximation for mixed-domain Gaussian processes, enabling efficient modeling and reduction of complex longitudinal data with categorical and continuous variables.

Contribution

It presents a novel linear-scaling approximation scheme for non-continuous covariance functions in mixed-domain GPs, applicable to Bayesian regression and model reduction.

Findings

The approach scales linearly with data size and basis functions.

It accurately approximates exact GP models at a fraction of the computational cost.

The method facilitates interpretable model reduction for large predictor sets.

Abstract

Gaussian process (GP) models that combine both categorical and continuous input variables have found use in analysis of longitudinal data and computer experiments. However, standard inference for these models has the typical cubic scaling, and common scalable approximation schemes for GPs cannot be applied since the covariance function is non-continuous. In this work, we derive a basis function approximation scheme for mixed-domain covariance functions, which scales linearly with respect to the number of observations and total number of basis functions. The proposed approach is naturally applicable to also Bayesian GP regression with discrete observation models. We demonstrate the scalability of the approach and compare model reduction techniques for additive GP models in a longitudinal data context. We confirm that we can approximate the exact GP model accurately in a fraction of the runtime compared to fitting the corresponding exact model. In addition, we demonstrate a scalable model reduction workflow for obtaining smaller and more interpretable models when dealing with a large number of candidate predictors.