VC-PCR: A Prediction Method based on Supervised Variable Selection and Clustering

TL;DR

VC-PCR is a novel supervised method that improves prediction accuracy, variable selection, and clustering in sparse linear models with clustered variables, outperforming existing methods.

Contribution

It introduces VC-PCR, a new approach that simultaneously supervises variable selection and clustering to enhance sparse linear prediction models.

Findings

VC-PCR outperforms competitors in prediction accuracy.

It achieves better variable selection in clustered data.

VC-PCR improves clustering performance when variables have a cluster structure.

Abstract

Sparse linear prediction methods suffer from decreased prediction accuracy when the predictor variables have cluster structure (e.g. there are highly correlated groups of variables). To improve prediction accuracy, various methods have been proposed to identify variable clusters from the data and integrate cluster information into a sparse modeling process. But none of these methods achieve satisfactory performance for prediction, variable selection and variable clustering simultaneously. This paper presents Variable Cluster Principal Component Regression (VC-PCR), a prediction method that supervises variable selection and variable clustering in order to solve this problem. Experiments with real and simulated data demonstrate that, compared to competitor methods, VC-PCR achieves better prediction, variable selection and clustering performance when cluster structure is present.