Debiased Bayesian Inference for High-dimensional Regression Models

TL;DR

This paper introduces a debiasing method for Bayesian posteriors in high-dimensional regression, ensuring valid frequentist inference and confidence sets, supported by theoretical guarantees and empirical demonstrations.

Contribution

It proposes a novel debiasing technique for Bayesian inference in high-dimensional models, with a new Bernstein-von Mises theorem ensuring frequentist validity.

Findings

Debiased posterior achieves valid confidence sets.

Method performs well in simulations.

Effective in economic applications.

Abstract

There has been significant progress in Bayesian inference based on sparsity-inducing (e.g., spike-and-slab and horseshoe-type) priors for high-dimensional regression models. The resulting posteriors, however, in general do not possess desirable frequentist properties, and the credible sets thus cannot serve as valid confidence sets even asymptotically. We introduce a novel debiasing approach that corrects the bias for the entire Bayesian posterior distribution. We establish a new Bernstein-von Mises theorem that guarantees the frequentist validity of the debiased posterior. We demonstrate the practical performance of our proposal through Monte Carlo simulations and two empirical applications in economics.