Estimation and inference for high-dimensional non-sparse models

TL;DR

This paper introduces a semiparametric re-modeling approach for high-dimensional linear regression that remains effective even when the model is non-sparse, improving estimation and prediction accuracy.

Contribution

It proposes a novel semiparametric inference method that is robust to non-sparsity, using instrumental variables and nonparametric adjustments after initial model selection.

Findings

Significant improvement in estimation accuracy for high-dimensional, non-sparse models.

Method performs comparably to classical sparse-model methods when the true model is sparse.

Robustness demonstrated through simulation studies with large p relative to n.

Abstract

To successfully work on variable selection, sparse model structure has become a basic assumption for all existing methods. However, this assumption is questionable as it is hard to hold in most of cases and none of existing methods may provide consistent estimation and accurate model prediction in nons-parse scenarios. In this paper, we propose semiparametric re-modeling and inference when the linear regression model under study is possibly non-sparse. After an initial working model is selected by a method such as the Dantzig selector adopted in this paper, we re-construct a globally unbiased semiparametric model by use of suitable instrumental variables and nonparametric adjustment. The newly defined model is identifiable, and the estimator of parameter vector is asymptotically normal. The consistency, together with the re-built model, promotes model prediction. This method naturally works when the model is indeed sparse and thus is of robustness against non-sparseness in certain sense. Simulation studies show that the new approach has, particularly when $p$ is much larger than $n$, significant improvement of estimation and prediction accuracies over the Gaussian Dantzig selector and other classical methods. Even when the model under study is sparse, our method is also comparable to the existing methods designed for sparse models.