Bayesian Global-Local Shrinkage with Univariate Guidance for Ultra-High-Dimensional Regression

TL;DR

The paper introduces BUGS, a Bayesian sparse regression method that uses univariate guidance for adaptive shrinkage, improving signal detection and scalability in ultra-high-dimensional data.

Contribution

It develops a novel guidance-informed prior and an active-set MCMC algorithm, enabling scalable Bayesian inference with theoretical guarantees and superior empirical performance.

Findings

Achieves strong signal recovery and false discovery control.

Scales to p=1,000,000 features, demonstrated on DNA methylation data.

Provides theoretical guarantees including posterior contraction and sure screening.

Abstract

We propose Bayesian Univariate-Guided Sparse Regression (BUGS), a novel global-local shrinkage framework that incorporates marginal association information directly into the prior through a continuous modulation of shrinkage. Unlike existing approaches that treat predictors symmetrically or rely on post hoc screening, BUGS embeds univariate guidance within the nonlinear variance structure of a regularized horseshoe prior, inducing adaptive shrinkage that enhances signal-noise separation. We establish theoretical guarantees including prior concentration, posterior contraction, and guidance-induced shrinkage separation, while demonstrating robustness under uninformative guidance. To enable scalability in ultra-high dimensions, we develop BUGS-Active, an active-set MCMC approximation that restricts local updates to a data-adaptive subset A_n, reducing per-iteration complexity from O(p) to O(|A_n|) while preserving key theoretical properties such as sure screening and contraction. Empirically, the proposed framework achieves strong signal recovery together with substantially improved control of false discovery rates relative to existing methods. BUGS-Active scales to dimensions up to p = 1,000,000, and is applied to a DNA methylation study with n=1051 subjects and approximately 850,000 CpG sites, yielding accurate prediction and interpretable sparse selection. These results establish marginally guided shrinkage as a powerful and scalable paradigm for high-dimensional Bayesian inference.