Nonparametric Canonical Correlation Analysis

TL;DR

This paper revisits Lancaster's classical theory to develop a practical, nonparametric CCA method that estimates correlations directly from data without heavy computation, outperforming kernel CCA and matching deep CCA.

Contribution

It introduces a novel nonparametric CCA algorithm based on density estimation and SVD, avoiding kernel matrix inversion and improving efficiency and performance.

Findings

NCCA performs better than kernel CCA in experiments.

NCCA is faster and more memory-efficient than kernel CCA.

PLCCA effectively combines linear and nonparametric views.

Abstract

Canonical correlation analysis (CCA) is a classical representation learning technique for finding correlated variables in multi-view data. Several nonlinear extensions of the original linear CCA have been proposed, including kernel and deep neural network methods. These approaches seek maximally correlated projections among families of functions, which the user specifies (by choosing a kernel or neural network structure), and are computationally demanding. Interestingly, the theory of nonlinear CCA, without functional restrictions, had been studied in the population setting by Lancaster already in the 1950s, but these results have not inspired practical algorithms. We revisit Lancaster's theory to devise a practical algorithm for nonparametric CCA (NCCA). Specifically, we show that the solution can be expressed in terms of the singular value decomposition of a certain operator associated with the joint density of the views. Thus, by estimating the population density from data, NCCA reduces to solving an eigenvalue system, superficially like kernel CCA but, importantly, without requiring the inversion of any kernel matrix. We also derive a partially linear CCA (PLCCA) variant in which one of the views undergoes a linear projection while the other is nonparametric. Using a kernel density estimate based on a small number of nearest neighbors, our NCCA and PLCCA algorithms are memory-efficient, often run much faster, and perform better than kernel CCA and comparable to deep CCA.