Regularization for supervised learning via the "hubNet" procedure

TL;DR

HubNet is a novel regularization method that models predictor connections via a hub-based graphical model, improving prediction accuracy and providing insights into predictor structure across various supervised learning tasks.

Contribution

Introduces HubNet, a new regularization approach that estimates predictor connections with a graphical model, applicable to multiple supervised learning methods and demonstrating improved performance.

Findings

Can outperform lasso in prediction accuracy

Provides insights into predictor network structure

Applicable to diverse supervised learning models

Abstract

We propose a new method for supervised learning. The hubNet procedure fits a hub-based graphical model to the predictors, to estimate the amount of "connection" that each predictor has with other predictors. This yields a set of predictor weights that are then used in a regularized regression such as the lasso or elastic net. The resulting procedure is easy to implement, can sometimes yields higher prediction accuracy that the lasso, and can give insights into the underlying structure of the predictors. HubNet can also be generalized seamlessly to other supervised problems such as regularized logistic regression (and other GLMs), Cox's proportional hazards model, and nonlinear procedures such as random forests and boosting. We prove some recovery results under a specialized model and illustrate the method on real and simulated data.