Group least squares regression for linear models with strongly correlated predictor variables

TL;DR

This paper introduces a group least squares regression method to effectively analyze strongly correlated predictor variables within groups, improving interpretability and estimation accuracy over traditional methods.

Contribution

It proposes a novel group-based least squares approach for multicollinear predictors, providing theoretical characterization and demonstrating advantages over existing techniques.

Findings

Effective estimation of group effects in strongly correlated variables

Advantages over ridge regression in handling multicollinearity

Insights into prediction accuracy and generalized linear models

Abstract

Traditionally, the least squares regression is mainly concerned with studying the effects of individual predictor variables, but strongly correlated variables generate multicollinearity which makes it difficult to study their effects. Existing methods for handling multicollinearity such as ridge regression are complicated. To resolve the multicollinearity issue without abandoning the simple least squares regression, for situations where predictor variables are in groups with strong within-group correlations but weak between-group correlations, we propose to study the effects of the groups with a group approach to the least squares regression. Using an all positive correlations arrangement of the strongly correlated variables, we first characterize group effects that are meaningful and can be accurately estimated. We then present the group approach with numerical examples and demonstrate its advantages over existing methods for handling multicollinearity. We also address a common misconception about prediction accuracy of the least squares estimated model and discuss through an example similar group effects in generalized linear models.