$\ell_1$-Regularized Generalized Least Squares

TL;DR

This paper introduces an $ ext{L}_1$-regularized GLS estimator for high-dimensional regressions with autocorrelated errors, combining LASSO and autoregressive modeling to improve estimation accuracy in the presence of autocorrelation.

Contribution

The paper proposes a feasible three-step $ ext{L}_1$-regularized GLS method that estimates autocorrelation structure and enhances high-dimensional regression accuracy.

Findings

The proposed estimator has smaller error than standard LASSO with autocorrelated errors.

Simulation results show the method performs comparably to LASSO with white noise and better with autocorrelation.

Theoretical analysis confirms the estimator's robustness in sub-Gaussian settings.

Abstract

We study an $\ell_{1}$-regularized generalized least-squares (GLS) estimator for high-dimensional regressions with autocorrelated errors. Specifically, we consider the case where errors are assumed to follow an autoregressive process, alongside a feasible variant of GLS that estimates the structure of this process in a data-driven manner. The estimation procedure consists of three steps: performing a LASSO regression, fitting an autoregressive model to the realized residuals, and then running a second-stage LASSO regression on the rotated (whitened) data. We examine the theoretical performance of the method in a sub-Gaussian random-design setting, in particular assessing the impact of the rotation on the design matrix and how this impacts the estimation error of the procedure. We show that our proposed estimators maintain smaller estimation error than an unadjusted LASSO regression when the errors are driven by an autoregressive process. A simulation study verifies the performance of the proposed method, demonstrating that the penalized (feasible) GLS-LASSO estimator performs on par with the LASSO in the case of white noise errors, whilst outperforming when the errors exhibit significant autocorrelation.