Group Model Selection Using Marginal Correlations: The Good, the Bad and the Ugly

TL;DR

This paper introduces GroTh, a simple yet effective method for high-dimensional group model selection based on marginal correlations, overcoming limitations of existing approaches and providing verifiable performance guarantees.

Contribution

It proposes a low-complexity thresholding approach for group selection that relates its performance to a verifiable predictor property, applicable to arbitrary predictors and coefficients.

Findings

GroTh effectively identifies relevant groups in high-dimensional settings.

Performance is linked to a verifiable predictor property.

The approach works with arbitrary predictors and coefficients.

Abstract

Group model selection is the problem of determining a small subset of groups of predictors (e.g., the expression data of genes) that are responsible for majority of the variation in a response variable (e.g., the malignancy of a tumor). This paper focuses on group model selection in high-dimensional linear models, in which the number of predictors far exceeds the number of samples of the response variable. Existing works on high-dimensional group model selection either require the number of samples of the response variable to be significantly larger than the total number of predictors contributing to the response or impose restrictive statistical priors on the predictors and/or nonzero regression coefficients. This paper provides comprehensive understanding of a low-complexity approach to group model selection that avoids some of these limitations. The proposed approach, termed Group Thresholding (GroTh), is based on thresholding of marginal correlations of groups of predictors with the response variable and is reminiscent of existing thresholding-based approaches in the literature. The most important contribution of the paper in this regard is relating the performance of GroTh to a polynomial-time verifiable property of the predictors for the general case of arbitrary (random or deterministic) predictors and arbitrary nonzero regression coefficients.