Multi-context principal component analysis

TL;DR

MCPCA is a new framework extending PCA to identify shared and context-specific factors in multi-context data, demonstrated on gene expression and language models.

Contribution

We introduce MCPCA, a novel theoretical and algorithmic approach for decomposing data into shared and unique factors across multiple contexts.

Findings

Reveals shared axes of variation across cancer types in gene expression data.

Identifies axes associated with lung cancer progression.

Maps stages of debate in language models over decades.

Abstract

Principal component analysis (PCA) is a tool to capture factors that explain variation in data. Across domains, data are now collected across multiple contexts (for example, individuals with different diseases, cells of different types, or words across texts). While the factors explaining variation in data are undoubtedly shared across subsets of contexts, no tools currently exist to systematically recover such factors. We develop multi-context principal component analysis (MCPCA), a theoretical and algorithmic framework that decomposes data into factors shared across subsets of contexts. Applied to gene expression, MCPCA reveals axes of variation shared across subsets of cancer types and an axis whose variability in tumor cells, but not mean, is associated with lung cancer progression. Applied to contextualized word embeddings from language models, MCPCA maps stages of a debate on human nature, revealing a discussion between science and fiction over decades. These axes are not found by combining data across contexts or by restricting to individual contexts. MCPCA is a principled generalization of PCA to address the challenge of understanding factors underlying data across contexts.