Distributed Estimation and Gap-Free Analysis of Canonical Correlations

TL;DR

This paper introduces a communication-efficient distributed algorithm for canonical correlation analysis that achieves optimal convergence rates without requiring a gap between canonical correlations, supported by extensive simulations and real data applications.

Contribution

It presents a novel multi-round distributed CCA algorithm with gap-free analysis, improving efficiency and removing restrictive assumptions of prior methods.

Findings

Achieves the same convergence rate as pooled data analysis.

Does not require an explicit gap between canonical correlations.

Demonstrates strong empirical performance on benchmark image data.

Abstract

Massive data analysis calls for distributed algorithms and theories. We design a multi-round distributed algorithm for canonical correlation analysis. We construct principal directions through the convex formulation of canonical correlation analysis and use the shift-and-invert preconditioning iteration to expedite the convergence rate. This distributed algorithm is communication-efficient. The resultant estimate achieves the same convergence rate as if all observations were pooled together, but does not impose stringent restrictions on the number of machines. We take a gap-free analysis to bypass the widely used yet unrealistic assumption of an explicit gap between the successive canonical correlations in the canonical correlation analysis. Extensive simulations and applications to three benchmark image data are conducted to demonstrate the empirical performance of our proposed algorithms and theories.