Unified Bayesian theory of sparse linear regression with nuisance parameters

TL;DR

This paper develops a Bayesian framework for high-dimensional sparse linear regression with nuisance parameters, establishing model selection consistency and credible set coverage guarantees.

Contribution

It introduces a unified Bayesian approach that handles various types of nuisance parameters and proves optimal posterior contraction and Bernstein-von Mises results.

Findings

Strong model selection consistency achieved.

Credible sets have guaranteed frequentist coverage.

Applicable to finite, high-, and infinite-dimensional nuisance parameters.

Abstract

We study frequentist asymptotic properties of Bayesian procedures for high-dimensional Gaussian sparse regression when unknown nuisance parameters are involved. Nuisance parameters can be finite-, high-, or infinite-dimensional. A mixture of point masses at zero and continuous distributions is used for the prior distribution on sparse regression coefficients, and appropriate prior distributions are used for nuisance parameters. The optimal posterior contraction of sparse regression coefficients, hampered by the presence of nuisance parameters, is also examined and discussed. It is shown that the procedure yields strong model selection consistency. A Bernstein-von Mises-type theorem for sparse regression coefficients is also obtained for uncertainty quantification through credible sets with guaranteed frequentist coverage. Asymptotic properties of numerous examples are investigated using the theories developed in this study.