Cell-JEPA: Latent Representation Learning for Single-Cell Transcriptomics

TL;DR

Cell-JEPA introduces a novel latent space prediction approach for single-cell transcriptomics, improving cell-type clustering robustness and addressing noise issues inherent in traditional reconstruction methods.

Contribution

The paper presents Cell-JEPA, a joint-embedding predictive architecture that enhances noise robustness and improves clustering performance in single-cell data analysis.

Findings

Cell-JEPA achieves 0.72 AvgBIO in zero-shot transfer, outperforming scGPT.

Predicting cell embeddings from partial data improves dropout-robust features.

Representation learning and perturbation modeling complement each other in cellular prediction.

Abstract

Single-cell foundation models learn by reconstructing masked gene expression, implicitly treating technical noise as signal. With dropout rates exceeding 90%, reconstruction objectives encourage models to encode measurement artifacts rather than stable cellular programs. We introduce Cell-JEPA, a joint-embedding predictive architecture that shifts learning from reconstructing sparse counts to predicting in latent space. The key insight is that cell identity is redundantly encoded across genes. We show predicting cell-level embeddings from partial observations forces the model to learn dropout-robust features. On cell-type clustering, Cell-JEPA achieves 0.72 AvgBIO in zero-shot transfer versus 0.53 for scGPT, a 36% relative improvement. On perturbation prediction within a single cell line, Cell-JEPA improves absolute-state reconstruction but not effect-size estimation, suggesting that representation learning and perturbation modeling address complementary aspects of cellular prediction.