Optimal Bias-Correction and Valid Inference in High-Dimensional Ridge Regression: A Closed-Form Solution

TL;DR

This paper introduces a novel bias-correction method for high-dimensional ridge regression that achieves valid inference and optimal bias mitigation for both p<n and p>n scenarios, supported by theoretical and empirical validation.

Contribution

It presents a new iterative bias-correction strategy and a Ridge-Screening method for effective bias mitigation and variable selection in high-dimensional ridge regression.

Findings

The proposed method achieves asymptotic unbiasedness in high-dimensional settings.

It provides valid inference procedures for both p<n and p>n cases.

Empirical results confirm improved bias correction and model selection performance.

Abstract

Ridge regression is an indispensable tool in big data analysis. Yet its inherent bias poses a significant and longstanding challenge, compromising both statistical efficiency and scalability across various applications. To tackle this critical issue, we introduce an iterative strategy to correct bias effectively when the dimension $p$ is less than the sample size $n$. For $p>n$, our method optimally mitigates the bias such that any remaining bias in the proposed de-biased estimator is unattainable through linear transformations of the response data. To address the remaining bias when $p>n$, we employ a Ridge-Screening (RS) method, producing a reduced model suitable for bias correction. Crucially, under certain conditions, the true model is nested within our selected one, highlighting RS as a novel variable selection approach. Through rigorous analysis, we establish the asymptotic properties and valid inferences of our de-biased ridge estimators for both $p<n$ and $p>n$, where, both $p$ and $n$ may increase towards infinity, along with the number of iterations. We further validate these results using simulated and real-world data examples. Our method offers a transformative solution to the bias challenge in ridge regression inferences across various disciplines.