# Tensor Canonical Correlation Analysis with Convergence and Statistical Guarantees

## TL;DR

This paper develops a tensor canonical correlation analysis framework with convergence guarantees, statistical bounds, and practical algorithms, enabling effective analysis of high-dimensional tensor data in large-scale applications.

## Contribution

It introduces a novel tensor CCA method with convergence and statistical guarantees, including scalable algorithms and initialization strategies for real-world tensor data analysis.

## Key findings

- Convergence of the power method for tensor CCA is established.
- The proposed method is effective on gene expression and air pollution data.
- Scalable algorithms with stochastic gradient descent are validated.

## Abstract

In many applications, such as classification of images or videos, it is of interest to develop a framework for tensor data instead of an ad-hoc way of transforming data to vectors due to the computational and under-sampling issues. In this paper, we study convergence and statistical properties of two-dimensional canonical correlation analysis \citep{Lee2007Two} under an assumption that data come from a probabilistic model. We show that carefully initialized the power method converges to the optimum and provide a finite sample bound. Then we extend this framework to tensor-valued data and propose the higher-order power method, which is commonly used in tensor decomposition, to extract the canonical directions. Our method can be used effectively in a large-scale data setting by solving the inner least squares problem with a stochastic gradient descent, and we justify convergence via the theory of Lojasiewicz's inequalities without any assumption on data generating process and initialization. For practical applications, we further develop (a) an inexact updating scheme which allows us to use the state-of-the-art stochastic gradient descent algorithm, (b) an effective initialization scheme which alleviates the problem of local optimum in non-convex optimization, and (c) a deflation procedure for extracting several canonical components. Empirical analyses on challenging data including gene expression and air pollution indexes in Taiwan, show the effectiveness and efficiency of the proposed methodology. Our results fill a missing, but crucial, part in the literature on tensor data.