Beyond the Flat-Spike: Adaptive Sparse CCA for Decaying and Unbalanced Signals

TL;DR

This paper introduces Bi-SEP, an adaptive algorithm for Sparse Canonical Correlation Analysis that leverages structured energy decay in signals to improve sample complexity bounds.

Contribution

It proposes a novel stagewise adaptive method, Bi-SEP, that exploits signal structure to overcome computational-statistical gaps in SCCA.

Findings

Bi-SEP achieves optimal sample complexity under power-law decay models.

A synergistic phase transition allows highly concentrated signals in one view to compensate for flat signals in the other.

Numerical experiments confirm the theoretical advantages of Bi-SEP in structured signal regimes.

Abstract

Sparse Canonical Correlation Analysis (SCCA) is a fundamental statistical tool for identifying linear relationships in high-dimensional, multi-view data. While minimax theory establishes an optimal sample complexity scaling additively with the sparsity levels of the canonical vectors, computationally efficient algorithms typically suffer from a suboptimal multiplicative dependence. This computational-statistical gap is intrinsically tied to worst-case ``flat'' signal assumptions. In practice, however, multi-view signals frequently exhibit structured energy concentration, such as a power-law decay. To exploit this structural concentration and bypass the worst-case bottleneck, we propose Bilateral Spectral Energy Pursuit (Bi-SEP). Operating directly on the cross-covariance matrix, Bi-SEP is a stagewise adaptive algorithm that utilizes a proxy refinement step to dynamically track and capture cross-view signal energy. Theoretically, we establish a profile-adaptive sample complexity bound governed by the coupled energy profiles of the two views. Notably, under power-law decay models, we reveal a synergistic phase transition: the optimal linear sample complexity is attainable provided that the aggregate decay rate of the two views is sufficiently large. This result demonstrates that a highly concentrated signal in one view allows the model to accommodate a completely flat signal in its partner. Numerical experiments validate our theoretical findings, illustrating the advantages of Bi-SEP in structured, non-flat signal regimes.