Adaptive Bayesian Multivariate Spline Knot Inference with Prior Specifications on Model Complexity

TL;DR

This paper introduces a Bayesian method for inferring the number and locations of knots in multivariate spline regression, addressing limitations of existing approaches by incorporating model complexity priors and using reversible jump MCMC.

Contribution

It proposes a novel prior on knot number and an analytic formula for the normal case, improving knot inference accuracy over traditional BIC-based methods.

Findings

Effective in function fitting with discontinuities

Accurately estimates knot number and locations

Demonstrates superior performance in manifold denoising

Abstract

In multivariate spline regression, the number and locations of knots influence the performance and interpretability significantly. However, due to non-differentiability and varying dimensions, there is no desirable frequentist method to make inference on knots. In this article, we propose a fully Bayesian approach for knot inference in multivariate spline regression. The existing Bayesian method often uses BIC to calculate the posterior, but BIC is too liberal and it will heavily overestimate the knot number when the candidate model space is large. We specify a new prior on the knot number to take into account the complexity of the model space and derive an analytic formula in the normal model. In the non-normal cases, we utilize the extended Bayesian information criterion to approximate the posterior density. The samples are simulated in the space with differing dimensions via reversible jump Markov chain Monte Carlo. We apply the proposed method in knot inference and manifold denoising. Experiments demonstrate the splendid capability of the algorithm, especially in function fitting with jumping discontinuity.