Sparse additive Gaussian process with soft interactions

TL;DR

This paper introduces a Bayesian additive regression model that employs a novel combination of shrinkages to efficiently achieve sparsity in both the number of components and variables, improving interpretability and scalability.

Contribution

It develops a new Bayesian additive regression framework with hard and soft shrinkages to control sparsity at multiple levels, addressing computational challenges in high-dimensional settings.

Findings

Effective variable selection demonstrated in simulations

Accurate interaction network estimation in real data

Superior performance compared to existing methods

Abstract

Additive nonparametric regression models provide an attractive tool for variable selection in high dimensions when the relationship between the response and predictors is complex. They offer greater flexibility compared to parametric non-linear regression models and better interpretability and scalability than the non-parametric regression models. However, achieving sparsity simultaneously in the number of nonparametric components as well as in the variables within each nonparametric component poses a stiff computational challenge. In this article, we develop a novel Bayesian additive regression model using a combination of hard and soft shrinkages to separately control the number of additive components and the variables within each component. An efficient algorithm is developed to select the importance variables and estimate the interaction network. Excellent performance is obtained in simulated and real data examples.