scHyena: Foundation Model for Full-Length Single-Cell RNA-Seq Analysis in Brain

TL;DR

scHyena is a novel Transformer-based foundation model that processes full-length single-cell RNA-seq data to improve cell type classification and imputation accuracy in brain tissue analysis.

Contribution

Introduces scHyena, a new Transformer architecture with a Hyena operator and gene-embedding, enabling full-length data processing and improved analysis of brain scRNA-seq data.

Findings

Outperforms benchmark methods in cell type classification.

Achieves superior results in scRNA-seq imputation.

Learns generalizable features of cells and genes.

Abstract

Single-cell RNA sequencing (scRNA-seq) has made significant strides in unraveling the intricate cellular diversity within complex tissues. This is particularly critical in the brain, presenting a greater diversity of cell types than other tissue types, to gain a deeper understanding of brain function within various cellular contexts. However, analyzing scRNA-seq data remains a challenge due to inherent measurement noise stemming from dropout events and the limited utilization of extensive gene expression information. In this work, we introduce scHyena, a foundation model designed to address these challenges and enhance the accuracy of scRNA-seq analysis in the brain. Specifically, inspired by the recent Hyena operator, we design a novel Transformer architecture called singe-cell Hyena (scHyena) that is equipped with a linear adaptor layer, the positional encoding via gene-embedding, and a {bidirectional} Hyena operator. This enables us to process full-length scRNA-seq data without losing any information from the raw data. In particular, our model learns generalizable features of cells and genes through pre-training scHyena using the full length of scRNA-seq data. We demonstrate the superior performance of scHyena compared to other benchmark methods in downstream tasks, including cell type classification and scRNA-seq imputation.