The Generalized Elastic Net for least squares regression with network-aligned signal and correlated design

TL;DR

This paper introduces the Generalized Elastic Net, a new regularization method for graph-structured regression that leverages network alignment and correlated Gaussian design to improve prediction and estimation accuracy.

Contribution

The paper proposes a novel $ ext{ extonehalf}$-penalty called the Generalized Elastic Net, extending elastic net regularization to graph-structured data with correlated design.

Findings

Derived graph-dependent error bounds for the estimator.

Developed a coordinate descent algorithm for large-scale problems.

Demonstrated improved performance over existing methods on real and synthetic datasets.

Abstract

We propose a novel $\ell_1+\ell_2$-penalty, which we refer to as the Generalized Elastic Net, for regression problems where the feature vectors are indexed by vertices of a given graph and the true signal is believed to be smooth or piecewise constant with respect to this graph. Under the assumption of correlated Gaussian design, we derive upper bounds for the prediction and estimation errors, which are graph-dependent and consist of a parametric rate for the unpenalized portion of the regression vector and another term that depends on our network alignment assumption. We also provide a coordinate descent procedure based on the Lagrange dual objective to compute this estimator for large-scale problems. Finally, we compare our proposed estimator to existing regularized estimators on a number of real and synthetic datasets and discuss its potential limitations.