Grouping predictors via network-wide metrics

TL;DR

This paper introduces a novel network-based methodology for grouping predictors in high-dimensional regression models, improving interpretability and predictive power while addressing computational and statistical challenges.

Contribution

It proposes a new supervised grouping algorithm using network-wide metrics, along with a model-assisted bootstrap for uncertainty assessment, with proven theoretical properties.

Findings

The method accurately identifies predictor groups in high-dimensional data.

The bootstrap procedure reduces computational complexity.

Numerical experiments demonstrate improved grouping and prediction performance.

Abstract

When multitudes of features can plausibly be associated with a response, both privacy considerations and model parsimony suggest grouping them to increase the predictive power of a regression model. Specifically, the identification of groups of predictors significantly associated with the response variable eases further downstream analysis and decision-making. This paper proposes a new data analysis methodology that utilizes the high-dimensional predictor space to construct an implicit network with weighted edges %and weights on the edges to identify significant associations between the response and the predictors. Using a population model for groups of predictors defined via network-wide metrics, a new supervised grouping algorithm is proposed to determine the correct group, with probability tending to one as the sample size diverges to infinity. For this reason, we establish several theoretical properties of the estimates of network-wide metrics. A novel model-assisted bootstrap procedure that substantially decreases computational complexity is developed, facilitating the assessment of uncertainty in the estimates of network-wide metrics. The proposed methods account for several challenges that arise in the high-dimensional data setting, including (i) a large number of predictors, (ii) uncertainty regarding the true statistical model, and (iii) model selection variability. The performance of the proposed methods is demonstrated through numerical experiments, data from sports analytics, and breast cancer data.