Automatic Dynamic Relevance Determination for Gaussian process regression with high-dimensional functional inputs

TL;DR

This paper introduces ADRD, a novel method for automatic relevance determination in Gaussian process regression with high-dimensional functional inputs, improving prediction accuracy and interpretability while reducing tuning complexity.

Contribution

It generalizes relevance determination to high-dimensional functional inputs using the ALF framework, enabling smooth, dynamic relevance profiling with fewer parameters.

Findings

ADRD outperforms traditional dimension reduction methods.

Predictive power comparable to high-dimensional models.

Enforces smooth relevance profiles, avoiding erratic patterns.

Abstract

In the context of Gaussian process regression with functional inputs, it is common to treat the input as a vector. The parameter space becomes prohibitively complex as the number of functional points increases, effectively becoming a hindrance for automatic relevance determination in high-dimensional problems. Generalizing a framework for time-varying inputs, we introduce the asymmetric Laplace functional weight (ALF): a flexible, parametric function that drives predictive relevance over the index space. Automatic dynamic relevance determination (ADRD) is achieved with three unknowns per input variable and enforces smoothness over the index space. Additionally, we discuss a screening technique to assess under complete absence of prior and model information whether ADRD is reasonably consistent with the data. Such tool may serve for exploratory analyses and model diagnostics. ADRD is applied to remote sensing data and predictions are generated in response to atmospheric functional inputs. Fully Bayesian estimation is carried out to identify relevant regions of the functional input space. Validation is performed to benchmark against traditional vector-input model specifications. We find that ADRD outperforms models with input dimension reduction via functional principal component analysis. Furthermore, the predictive power is comparable to high-dimensional models, in terms of both mean prediction and uncertainty, with 10 times fewer tuning parameters. Enforcing smoothness on the predictive relevance profile rules out erratic patterns associated with vector-input models.