Canonical Correlation Analysis of Datasets with a Common Source Graph

TL;DR

This paper introduces a novel graph-regularized canonical correlation analysis (gCCA) that leverages source graph geometry to improve data analysis, especially in small-sample and nonlinear settings, with demonstrated benefits in image classification.

Contribution

It proposes a new gCCA method incorporating source graph information, along with dual and kernel formulations, advancing CCA's ability to exploit geometric data structures.

Findings

gCCA outperforms traditional CCA in classification tasks

Dual and kernel gCCA effectively handle small sample sizes and nonlinear dependencies

Graph regularization enhances the discovery of shared sources in datasets

Abstract

Canonical correlation analysis (CCA) is a powerful technique for discovering whether or not hidden sources are commonly present in two (or more) datasets. Its well-appreciated merits include dimensionality reduction, clustering, classification, feature selection, and data fusion. The standard CCA however, does not exploit the geometry of the common sources, which may be available from the given data or can be deduced from (cross-) correlations. In this paper, this extra information provided by the common sources generating the data is encoded in a graph, and is invoked as a graph regularizer. This leads to a novel graph-regularized CCA approach, that is termed graph (g) CCA. The novel gCCA accounts for the graph-induced knowledge of common sources, while minimizing the distance between the wanted canonical variables. Tailored for diverse practical settings where the number of data is smaller than the data vector dimensions, the dual formulation of gCCA is also developed. One such setting includes kernels that are incorporated to account for nonlinear data dependencies. The resultant graph-kernel (gk) CCA is also obtained in closed form. Finally, corroborating image classification tests over several real datasets are presented to showcase the merits of the novel linear, dual, and kernel approaches relative to competing alternatives.