$\ell_1$-norm constrained multi-block sparse canonical correlation analysis via proximal gradient descent

TL;DR

This paper introduces a proximal gradient descent method with an $ ext{l}_1$ constraint for high-dimensional multi-block CCA, providing theoretical guarantees and competitive empirical performance.

Contribution

It develops a novel $ ext{l}_1$-constrained approach for multi-block CCA with proven rate-optimality and an easy deflation procedure for multiple eigenvector estimation.

Findings

Method achieves rate-optimal estimates under certain conditions.

Competitive performance demonstrated in simulations and real data.

Provides theoretical understanding for $ ext{l}_1$ constrained multi-block CCA.

Abstract

Multi-block CCA constructs linear relationships explaining coherent variations across multiple blocks of data. We view the multi-block CCA problem as finding leading generalized eigenvectors and propose to solve it via a proximal gradient descent algorithm with $\ell_1$ constraint for high dimensional data. In particular, we use a decaying sequence of constraints over proximal iterations, and show that the resulting estimate is rate-optimal under suitable assumptions. Although several previous works have demonstrated such optimality for the $\ell_0$ constrained problem using iterative approaches, the same level of theoretical understanding for the $\ell_1$ constrained formulation is still lacking. We also describe an easy-to-implement deflation procedure to estimate multiple eigenvectors sequentially. We compare our proposals to several existing methods whose implementations are available on R CRAN, and the proposed methods show competitive performances in both simulations and a real data example.