Multivariate L\'evy Adaptive B-Spline Regression

TL;DR

This paper introduces MLABS, a Bayesian nonparametric regression model using Le9vy process priors and B-spline bases, capable of modeling complex functions with varying smoothness and high interaction orders.

Contribution

It develops a multivariate Le9vy adaptive B-spline regression model that automatically selects basis functions and adapts to function smoothness, improving predictive stability and accuracy.

Findings

MLABS performs comparably to state-of-the-art models on regression and classification tasks.

MLABS demonstrates superior stability and accuracy in low-dimensional data.

The model effectively captures functions with varying degrees of smoothness.

Abstract

We develop a fully Bayesian nonparametric regression model based on a L\'evy process prior named MLABS (Multivariate L\'evy Adaptive B-Spline regression) model, a multivariate version of the LARK (L\'evy Adaptive Regression Kernels) models, for estimating unknown functions with either varying degrees of smoothness or high interaction orders. L\'evy process priors have advantages of encouraging sparsity in the expansions and providing automatic selection over the number of basis functions. The unknown regression function is expressed as a weighted sum of tensor product of B-spline basis functions as the elements of an overcomplete system, which can deal with multi-dimensional data. The B-spline basis can express systematically functions with varying degrees of smoothness. By changing a set of degrees of the tensor product basis function, MLABS can adapt the smoothness of target functions due to the nice properties of B-spline bases. The local support of the B-spline basis enables the MLABS to make more delicate predictions than other existing methods in the two-dimensional surface data. Experiments on various simulated and real-world datasets illustrate that the MLABS model has comparable performance on regression and classification problems. We also show that the MLABS model has more stable and accurate predictive abilities than state-of-the-art nonparametric regression models in relatively low-dimensional data.