Causal Circuit Tracing Reveals Distinct Computational Architectures in Single-Cell Foundation Models: Inhibitory Dominance, Biological Coherence, and Cross-Model Convergence

TL;DR

This study introduces causal circuit tracing to analyze biological foundation models, revealing inhibitory dominance, biological coherence, and conserved computational structures across models, with implications for understanding disease-related features.

Contribution

The paper presents a novel causal circuit tracing method for biological models, demonstrating consistent inhibitory dominance and cross-model convergence across different architectures and cell types.

Findings

Models show ~53% biological coherence.

Inhibitory dominance ranges from 65% to 89%.

Cross-model analysis finds 1,142 conserved domain pairs.

Abstract

Motivation: Sparse autoencoders (SAEs) decompose foundation model activations into interpretable features, but causal feature-to-feature interactions across network depth remain unknown for biological foundation models. Results: We introduce causal circuit tracing by ablating SAE features and measuring downstream responses, and apply it to Geneformer V2-316M and scGPT whole-human across four conditions (96,892 edges, 80,191 forward passes). Both models show approximately 53 percent biological coherence and 65 to 89 percent inhibitory dominance, invariant to architecture and cell type. scGPT produces stronger effects (mean absolute d = 1.40 vs. 1.05) with more balanced dynamics. Cross-model consensus yields 1,142 conserved domain pairs (10.6x enrichment, p < 0.001). Disease-associated domains are 3.59x more likely to be consensus. Gene-level CRISPRi validation shows 56.4 percent directional accuracy, confirming co-expression rather than causal encoding.