Variable Selection Using Nearest Neighbor Gaussian Processes

TL;DR

This paper presents a scalable Bayesian variable selection method using nearest neighbor Gaussian processes, improving interpretability and regularization in regression models.

Contribution

It introduces a novel approach combining nearest neighbor Gaussian processes with Bayesian variable selection, including a new inference algorithm and practical evaluation.

Findings

Effective variable selection demonstrated on simulated data

Improved model interpretability and regularization

Scalable inference with a novel Metropolis-Within-Gibbs algorithm

Abstract

We introduce a novel Bayesian approach for variable selection using Gaussian process regression, which is crucial for enhancing interpretability and model regularization. Our method employs nearest neighbor Gaussian processes, serving as scalable approximations of classical Gaussian processes. Variable selection is achieved by conditioning the process mean and covariance function on a random set that represents the indices of contributing variables. A priori beliefs regarding this set control the variable selection, while reference priors are assigned to the remaining model parameters, ensuring numerical robustness in the process covariance matrix. We propose a Metropolis-Within-Gibbs algorithm for model inference. Evaluation using simulated data, a computer experiment approximation, and two real-world data sets demonstrate the effectiveness of our approach.