Graph-based Square-Root Estimation for Sparse Linear Regression

TL;DR

This paper introduces a novel graph-based square-root estimation model for sparse linear regression that effectively handles high-dimensional data, diverse noise types, and unknown noise standard deviation, with strong theoretical guarantees and computational efficiency.

Contribution

The paper proposes a general GSRE model that unifies several classic regression models, providing theoretical analysis and an efficient algorithm for high-dimensional sparse regression.

Findings

Outperforms existing methods in estimation accuracy

Demonstrates robustness to various noise types

Achieves efficient computation with ADMM

Abstract

Sparse linear regression is one of the classic problems in the field of statistics, which has deep connections and high intersections with optimization, computation, and machine learning. To address the effective handling of high-dimensional data, the diversity of real noise, and the challenges in estimating standard deviation of the noise, we propose a novel and general graph-based square-root estimation (GSRE) model for sparse linear regression. Specifically, we use square-root-loss function to encourage the estimators to be independent of the unknown standard deviation of the error terms and design a sparse regularization term by using the graphical structure among predictors in a node-by-node form. Based on the predictor graphs with special structure, we highlight the generality by analyzing that the model in this paper is equivalent to several classic regression models. Theoretically, we also analyze the finite sample bounds, asymptotic normality and model selection consistency of GSRE method without relying on the standard deviation of error terms. In terms of computation, we employ the fast and efficient alternating direction method of multipliers. Finally, based on a large number of simulated and real data with various types of noise, we demonstrate the performance advantages of the proposed method in estimation, prediction and model selection.