Dependency-Aware Shrinkage Priors for High Dimensional Regression

TL;DR

This paper introduces dependency-aware shrinkage priors for high-dimensional regression, exploring how modeling coefficient dependencies affects estimation and prediction, with theoretical analysis and empirical evaluation.

Contribution

It extends traditional shrinkage priors by incorporating correlation structures, providing insights into their impact on inference and performance.

Findings

Dependence modeling improves parameter recovery with correlated predictors.

Modest gains in predictive accuracy from dependence modeling.

Guidance on when to use dependence-aware priors based on inferential goals.

Abstract

In high dimensional regression, global local shrinkage priors have gained significant traction for their ability to yield sparse estimates, improve parameter recovery, and support accurate predictive modeling. While recent work has explored increasingly flexible shrinkage prior structures, the role of explicitly modeling dependencies among coefficients remains largely unexplored. In this paper, we investigate whether incorporating such structures into traditional shrinkage priors improves their performance. We introduce dependency-aware shrinkage priors, an extension of continuous shrinkage priors that integrates correlation structures inspired by Zellner's g prior approach. We provide theoretical insights into how dependence alters the prior and posterior structure, and evaluate the method empirically through simulations and real data. We find that modeling dependence can improve parameter recovery when predictors are strongly correlated, but offers only modest gains in predictive accuracy. These findings suggest that prior dependence should be used selectively and guided by the specific inferential goals of the analysis.