High-Dimensional Bayesian Inference in Nonparametric Additive Models

TL;DR

This paper introduces a Bayesian method for high-dimensional nonparametric additive models, enabling stochastic model search and parameter estimation even when the number of components exceeds the sample size.

Contribution

It proposes a fully Bayesian approach with new g-priors and an efficient MCMC algorithm for ultrahigh-dimensional additive models, with theoretical guarantees.

Findings

Posterior probability of the true model approaches one asymptotically.

New g-priors improve model flexibility and performance.

Simulation results demonstrate computational efficiency and effectiveness.

Abstract

A fully Bayesian approach is proposed for ultrahigh-dimensional nonparametric additive models in which the number of additive components may be larger than the sample size, though ideally the true model is believed to include only a small number of components. Bayesian approaches can conduct stochastic model search and fulfill flexible parameter estimation by stochastic draws. The theory shows that the proposed model selection method has satisfactory properties. For instance, when the hyperparameter associated with the model prior is correctly specified, the true model has posterior probability approaching one as the sample size goes to infinity; when this hyperparameter is incorrectly specified, the selected model is still acceptable since asymptotically it is proved to be nested in the true model. To enhance model flexibility, two new $g$-priors are proposed and their theoretical performance is examined. We also propose an efficient MCMC algorithm to handle the computational issues. Several simulation examples are provided to demonstrate the computational advantages of our method.