Sparse CCA: Adaptive Estimation and Computational Barriers

TL;DR

This paper advances the understanding of sparse canonical correlation analysis by establishing optimal estimation rates, proposing a computationally feasible estimator, and demonstrating fundamental computational barriers in high-dimensional settings.

Contribution

It introduces adaptive minimax rates for sparse CCA, proposes a practical estimator achieving these rates, and proves computational lower bounds linked to the Planted Clique problem.

Findings

Optimal minimax estimation rates for sparse CCA vectors.

A feasible estimator that attains these rates adaptively.

Computational lower bounds indicating hardness of consistent estimation.

Abstract

Canonical correlation analysis is a classical technique for exploring the relationship between two sets of variables. It has important applications in analyzing high dimensional datasets originated from genomics, imaging and other fields. This paper considers adaptive minimax and computationally tractable estimation of leading sparse canonical coefficient vectors in high dimensions. First, we establish separate minimax estimation rates for canonical coefficient vectors of each set of random variables under no structural assumption on marginal covariance matrices. Second, we propose a computationally feasible estimator to attain the optimal rates adaptively under an additional sample size condition. Finally, we show that a sample size condition of this kind is needed for any randomized polynomial-time estimator to be consistent, assuming hardness of certain instances of the Planted Clique detection problem. The result is faithful to the Gaussian models used in the paper. As a byproduct, we obtain the first computational lower bounds for sparse PCA under the Gaussian single spiked covariance model.