MAP model selection in Gaussian regression

TL;DR

This paper introduces a Bayesian model selection method for Gaussian linear regression that adapts to high-dimensional settings, providing theoretical guarantees of optimality and minimaxity under various design conditions.

Contribution

It develops a Bayesian approach linking model selection to penalized least squares, with conditions ensuring asymptotic minimaxity in diverse regression scenarios.

Findings

Establishes an oracle inequality for the estimator.

Proves asymptotic minimaxity under specific prior conditions.

Applicable to both sparse and dense high-dimensional models.

Abstract

We consider a Bayesian approach to model selection in Gaussian linear regression, where the number of predictors might be much larger than the number of observations. From a frequentist view, the proposed procedure results in the penalized least squares estimation with a complexity penalty associated with a prior on the model size. We investigate the optimality properties of the resulting estimator. We establish the oracle inequality and specify conditions on the prior that imply its asymptotic minimaxity within a wide range of sparse and dense settings for "nearly-orthogonal" and "multicollinear" designs.