# BayesRare: Bayesian mixture model for population-level rare cell type detection in multi-subject single-cell RNA sequencing data

## TL;DR

BayesRare is a new method that detects rare cell types across multiple scRNA-seq datasets, improving accuracy and uncovering disease-related patterns.

## Contribution

BayesRare introduces a hierarchical Bayesian framework for population-level rare cell detection in multi-subject scRNA-seq data.

## Key findings

- BayesRare achieves superior precision and reduces false positives in rare cell detection.
- The method uncovers biologically meaningful disease-specific rare subtypes across real datasets.
- BayesRare integrates cross-subject information to quantify uncertainty and infer group-level differences.

## Abstract

Rare cell types in single-cell RNA sequencing (scRNA-seq) data often encode essential biological signals, such as early disease markers or key immune regulators. With advancing technologies, large-scale scRNA-seq cohorts from multiple subjects now enable population-level analyses of the prevalence, heterogeneity, and disease associations of rare cell populations. However, existing methods for rare cell detection are typically limited to single datasets and cannot effectively leverage cross-subject information. To tackle this challenge, we present BayesRare, a hierarchical Bayesian framework for population-level rare cell discovery in multi-subject scRNA-seq data. The method augments a Bayesian mixture model with a rare cluster indicator, supporting joint cell-type clustering and rare-population identification. By explicitly characterizing the statistical properties of rare cell types, BayesRare integrates evidence across subjects, quantifies uncertainty via posterior probabilities, and enables inference of group-level differences (e.g. patients versus controls). Across synthetic and three real datasets, BayesRare achieves superior precision, reduces false positives, and uncovers biologically meaningful disease-specific rare subtypes. The R package of BayesRare is available at https://github.com/yinqiaoyan/BayesRare.