Efficient Bayesian Multivariate Surface Regression

TL;DR

This paper introduces an efficient Bayesian multivariate surface regression model that combines additive and interactive splines, utilizing a joint knot update MCMC algorithm and shrinkage priors to enhance predictive accuracy and computational efficiency.

Contribution

It proposes a novel Bayesian regression framework with joint knot estimation and adaptive shrinkage, improving flexibility and performance over traditional methods.

Findings

Model achieves high computational efficiency.

Allows for flexible adaptation to data nonlinearity.

Demonstrates improved out-of-sample prediction in simulations and real data.

Abstract

Methods for choosing a fixed set of knot locations in additive spline models are fairly well established in the statistical literature. While most of these methods are in principle directly extendable to non-additive surface models, they are less likely to be successful in that setting because of the curse of dimensionality, especially when there are more than a couple of covariates. We propose a regression model for a multivariate Gaussian response that combines both additive splines and interactive splines, and a highly efficient MCMC algorithm that updates all the knot locations jointly. We use shrinkage priors to avoid overfitting with different estimated shrinkage factors for the additive and surface part of the model, and also different shrinkage parameters for the different response variables. This makes it possible for the model to adapt to varying degrees of nonlinearity in different parts of the data in a parsimonious way. Simulated data and an application to firm leverage data show that the approach is computationally efficient, and that allowing for freely estimated knot locations can offer a substantial improvement in out-of-sample predictive performance.