Semiparametric Bayesian Information Criterion for Model Selection in Ultra-high Dimensional Additive Models

TL;DR

This paper extends the Bayesian information criterion (BIC) to semiparametric additive models in ultra-high dimensional settings, demonstrating its consistency and broader applicability beyond linear models.

Contribution

It develops a BIC-type criterion for additive models with diverging dimensions, providing theoretical guarantees and relaxing noise assumptions.

Findings

The BIC-type criterion is consistent for additive models with exponentially diverging dimensions.

Theoretical results are extended to more general noise conditions.

Applicability of BIC is broadened to semiparametric models.

Abstract

For linear models with a diverging number of parameters, it has recently been shown that modified versions of Bayesian information criterion (BIC) can identify the true model consistently. However, in many cases there is little justification that the effects of the covariates are actually linear. Thus a semiparametric model such as the additive model studied here, is a viable alternative. We demonstrate that theoretical results on the consistency of BIC-type criterion can be extended to this more challenging situation, with dimension diverging exponentially fast with sample size. Besides, the noise assumptions are relaxed in our theoretical studies. These efforts significantly enlarge the applicability of the criterion to a more general class of models.