On Sparse Canonical Correlation Analysis

TL;DR

This paper investigates Sparse Canonical Correlation Analysis (SCCA), offering new formulations and algorithms to improve interpretability and address computational challenges in high-dimensional data analysis.

Contribution

It introduces a combinatorial formulation and a mixed-integer semidefinite programming model for SCCA, along with complexity analysis and efficient algorithms.

Findings

Proposed approximation algorithms for SCCA.

Validated effectiveness through numerical experiments.

Established complexity results for special cases.

Abstract

The classical Canonical Correlation Analysis (CCA) identifies the correlations between two sets of multivariate variables based on their covariance, which has been widely applied in diverse fields such as computer vision, natural language processing, and speech analysis. Despite its popularity, CCA can encounter challenges in explaining correlations between two variable sets within high-dimensional data contexts. Thus, this paper studies Sparse Canonical Correlation Analysis (SCCA) that enhances the interpretability of CCA. We first show that SCCA generalizes three well-known sparse optimization problems, sparse PCA, sparse SVD, and sparse regression, which are all classified as NP-hard problems. This result motivates us to develop strong formulations and efficient algorithms. Our main contributions include (i) the introduction of a combinatorial formulation that captures the essence of SCCA and allows the development of approximation algorithms; (ii) the derivation of an equivalent mixed-integer semidefinite programming model that facilitates a specialized branch-and-cut algorithm with analytical cuts; and (iii) the establishment of the complexity results for two low-rank special cases of SCCA. The effectiveness of our proposed formulations and algorithms is validated through numerical experiments.