Bipartite Graph Attention-based Clustering for Large-scale scRNA-seq Data

TL;DR

This paper introduces BGFormer, a bipartite graph transformer model that uses learnable anchor tokens and bipartite attention to efficiently cluster large-scale scRNA-seq data, overcoming previous quadratic complexity limitations.

Contribution

The paper presents a novel bipartite graph attention mechanism with learnable anchors, enabling linear scalability for large scRNA-seq clustering tasks.

Findings

BGFormer achieves linear complexity with respect to cell number.

Experimental results show improved scalability and clustering accuracy.

The model effectively groups cells in large-scale datasets.

Abstract

scRNA-seq clustering is a critical task for analyzing single-cell RNA sequencing (scRNA-seq) data, as it groups cells with similar gene expression profiles. Transformers, as powerful foundational models, have been applied to scRNA-seq clustering. Their self-attention mechanism automatically assigns higher attention weights to cells within the same cluster, enhancing the distinction between clusters. Existing methods for scRNA-seq clustering, such as graph transformer-based models, treat each cell as a token in a sequence. Their computational and space complexities are $\mathcal{O}(n^2)$ with respect to the number of cells, limiting their applicability to large-scale scRNA-seq datasets.To address this challenge, we propose a Bipartite Graph Transformer-based clustering model (BGFormer) for scRNA-seq data. We introduce a set of learnable anchor tokens as shared reference points to represent the entire dataset. A bipartite graph attention mechanism is introduced to learn the similarity between cells and anchor tokens, bringing cells of the same class closer together in the embedding space. BGFormer achieves linear computational complexity with respect to the number of cells, making it scalable to large datasets. Experimental results on multiple large-scale scRNA-seq datasets demonstrate the effectiveness and scalability of BGFormer.