Grouping Strategies and Thresholding for High Dimensional Linear Models

TL;DR

This paper introduces the GR-LOL algorithm for high-dimensional linear models with structured sparsity, utilizing grouping and thresholding to improve estimation accuracy, with theoretical convergence guarantees and practical advantages over existing methods.

Contribution

The paper proposes a novel two-step block thresholding algorithm, GR-LOL, with data-driven grouping strategies and proven convergence rates, enhancing high-dimensional regression estimation.

Findings

GR-LOL outperforms standard LOL in practical tests

Grouping can significantly improve estimation accuracy

GR-LOL compares favorably with group-Lasso methods

Abstract

The estimation problem in a high regression model with structured sparsity is investigated. An algorithm using a two steps block thresholding procedure called GR-LOL is provided. Convergence rates are produced: they depend on simple coherence-type indices of the Gram matrix -easily checkable on the data- as well as sparsity assumptions of the model parameters measured by a combination of $l_1$ within-blocks with $l_q,q<1$ between-blocks norms. The simplicity of the coherence indicator suggests ways to optimize the rates of convergence when the group structure is not naturally given by the problem and is unknown. In such a case, an auto-driven procedure is provided to determine the regressors groups (number and contents). An intensive practical study compares our grouping methods with the standard LOL algorithm. We prove that the grouping rarely deteriorates the results but can improve them very significantly. GR-LOL is also compared with group-Lasso procedures and exhibits a very encouraging behavior. The results are quite impressive, especially when GR-LOL algorithm is combined with a grouping pre-processing.