Discovering Interpretable Biological Concepts in Single-cell RNA-seq Foundation Models

TL;DR

This paper introduces a new interpretability framework for single-cell RNA-seq models that uses attribution methods and ontology-driven analysis to uncover biologically meaningful concepts, aiding hypothesis generation.

Contribution

It presents a novel concept-based interpretability framework combining attribution with biological pathway enrichment for single-cell RNA-seq models.

Findings

Concepts improve interpretability over individual neurons.

Framework successfully interprets models trained on immune cell datasets.

Interpretations align with known biological pathways.

Abstract

Single-cell RNA-seq foundation models achieve strong performance on downstream tasks but remain black boxes, limiting their utility for biological discovery. Recent work has shown that sparse dictionary learning can extract concepts from deep learning models, with promising applications in biomedical imaging and protein models. However, interpreting biological concepts remains challenging, as biological sequences are not inherently human-interpretable. We introduce a novel concept-based interpretability framework for single-cell RNA-seq models with a focus on concept interpretation and evaluation. We propose an attribution method with counterfactual perturbations that identifies genes that influence concept activation, moving beyond correlational approaches like differential expression analysis. We then provide two complementary interpretation approaches: an expert-driven analysis facilitated by an interactive interface and an ontology-driven method with attribution-based biological pathway enrichment. Applying our framework to two well-known single-cell RNA-seq models from the literature, we interpret concepts extracted by Top-K Sparse Auto-Encoders trained on two immune cell datasets. With a domain expert in immunology, we show that concepts improve interpretability compared to individual neurons while preserving the richness and informativeness of the latent representations. This work provides a principled framework for interpreting what biological knowledge foundation models have encoded, paving the way for their use for hypothesis generation and discovery.