Stacked SVD or SVD stacked? A Random Matrix Theory perspective on data integration

TL;DR

This paper compares two data integration methods, Stack-SVD and SVD-Stack, using Random Matrix Theory to derive their asymptotic performance, revealing when and how to optimally weight and choose between them.

Contribution

The paper provides the first rigorous asymptotic analysis of Stack-SVD and SVD-Stack, deriving phase transitions and optimal weighting schemes for improved data integration.

Findings

Optimally weighted Stack-SVD outperforms SVD-Stack in asymptotic regimes.

Derived exact formulas for performance and phase transitions of both methods.

Provided practical algorithms for estimating optimal weights from data.

Abstract

Modern data analysis increasingly requires identifying shared latent structure across multiple high-dimensional datasets. A commonly used model assumes that the data matrices are noisy observations of low-rank matrices with a shared singular subspace. In this case, two primary methods have emerged for estimating this shared structure, which vary in how they integrate information across datasets. The first approach, termed Stack-SVD, concatenates all the datasets, and then performs a singular value decomposition (SVD). The second approach, termed SVD-Stack, first performs an SVD separately for each dataset, then aggregates the top singular vectors across these datasets, and finally computes a consensus amongst them. While these methods are widely used, they have not been rigorously studied in the proportional asymptotic regime, which is of great practical relevance in today's world of increasing data size and dimensionality. This lack of theoretical understanding has led to uncertainty about which method to choose and limited the ability to fully exploit their potential. To address these challenges, we derive exact expressions for the asymptotic performance and phase transitions of these two methods and develop optimal weighting schemes to further improve both methods. Our analysis reveals that while neither method uniformly dominates the other in the unweighted case, optimally weighted Stack-SVD dominates optimally weighted SVD-Stack. We extend our analysis to accommodate multiple shared components, and provide practical algorithms for estimating optimal weights from data, offering theoretical guidance for method selection in practical data integration problems. Extensive numerical simulations and semi-synthetic experiments on genomic data corroborate our theoretical findings.