Proximal Projection for Doubly Sparse Regularized Models

TL;DR

This paper introduces a novel regularization method for high-dimensional regression that leverages predictor graph structures, using proximal projection to efficiently optimize latent variables for improved sparsity and computational efficiency.

Contribution

It proposes a new regularization approach that decomposes coefficients into latent variables aligned with predictor graphs and introduces a proximal projection technique for efficient optimization.

Findings

Method shows stable performance across various graph structures.

Proximal projection conserves computational resources in high-dimensional settings.

Simulation and real data demonstrate improved efficiency and stability.

Abstract

Regularization is often used in high-dimensional regression settings to generate a sparse model, which can save tremendous computing resources and identify predictors that are most strongly associated with the response. When the predictors can be represented by a Gaussian graphical model, the structure of the predictor graph can be exploited during regularization. Our proposed model exploits this underlying predictor graph structure by decomposing the estimated coefficient vector into a sum of latent variables that correspond to the sum of each node contribution to the coefficient vector. Regularization is then performed on the latent variables rather than on the coefficient vector directly. We use a penalty function that permits a clear user-defined trade-off between the L1 and L2 penalties and propose a novel proximal projection during optimization. Further, our implementation computes the projection operator for the intersection of selected groups, which conserves more computing resources compared to predictor duplication methods, especially for high-dimensional data. Through simulation, we evaluate the performance of our approach under different graph structures and node counts, and present results on real-world data. Results suggest that our method exhibits stable performance relative to other singly or doubly sparse graphical regression models.