Grouping effects of sparse CCA models in variable selection

TL;DR

This paper investigates the grouping effects of standard and simplified sparse canonical correlation analysis models in variable selection, revealing that the simplified model tends to select entire groups of correlated variables together, unlike the standard model.

Contribution

The paper provides a theoretical analysis of the grouping behavior of both SCCA models in high-dimensional variable selection, supported by empirical validation.

Findings

Simplified SCCA selects entire variable groups together.

Standard SCCA randomly selects dominant variables within groups.

Empirical results confirm theoretical predictions.

Abstract

The sparse canonical correlation analysis (SCCA) is a bi-multivariate association model that finds sparse linear combinations of two sets of variables that are maximally correlated with each other. In addition to the standard SCCA model, a simplified SCCA criterion which maixmizes the cross-covariance between a pair of canonical variables instead of their cross-correlation, is widely used in the literature due to its computational simplicity. However, the behaviors/properties of the solutions of these two models remain unknown in theory. In this paper, we analyze the grouping effect of the standard and simplified SCCA models in variable selection. In high-dimensional settings, the variables often form groups with high within-group correlation and low between-group correlation. Our theoretical analysis shows that for grouped variable selection, the simplified SCCA jointly selects or deselects a group of variables together, while the standard SCCA randomly selects a few dominant variables from each relevant group of correlated variables. Empirical results on synthetic data and real imaging genetics data verify the finding of our theoretical analysis.