# A benchmark of semi-supervised scRNA-seq integration methods in real-world scenarios

## TL;DR

This paper benchmarks semi-supervised scRNA-seq integration methods under realistic conditions and finds that they offer limited advantages over unsupervised methods when labels are imperfect.

## Contribution

The first systematic benchmark of semi-supervised scRNA-seq integration methods under diverse, realistic annotation scenarios.

## Key findings

- Semi-supervised methods like scDREAMER perform well with perfect labels but degrade rapidly with imperfect ones.
- Only scANVI and ssSTACAS maintain stable performance under realistic label imperfections.
- Unsupervised methods like scCRAFT are recommended as the most reliable default when labels are incomplete or noisy.

## Abstract

Semi-supervised methods for single-cell RNA-seq integration promise improved batch correction and preservation of biological signal by leveraging cell-type labels. However, reported benefits and robustness of them towards imperfect cell type labels often come from overly idealized settings. Here we present, to our knowledge, the first systematic benchmark comparing leading semi-supervised methods with widely used unsupervised approaches across six diverse datasets under realistic conditions. Beyond randomly missing or erroneous labels, we examine four additional scenarios (boundary-mixed labels, batch-specific annotations, auto-generated labels and varied-granularity labels) and evaluate performance using nine established metrics. We find that although semi-supervised methods can provide benefits under perfect annotations, their robustness often degrades substantially under realistic imperfections. Only scANVI and ssSTACAS maintain stable but modest improvements over their unsupervised counterparts, and none consistently outperform the strongest unsupervised approach. These results indicate that current semi-supervised strategies offer limited practical advantage when label quality is modest uncertain.

Single-cell RNA sequencing (scRNA-seq) offers deep insights into cellular diversity, yet combining datasets from different sources remains difficult due to technical “batch effects.” While semi-supervised integration methods promise to improve alignment by utilizing cell-type labels, their reliability under realistic, imperfect conditions remains unproven. We performed a comprehensive benchmark comparing leading semi-supervised algorithms against standard unsupervised approaches using realistic scenarios, such as randomly missing, mixing at edge, partially annotated by batches, automated labeling and varied-granularity labels. Our results overturn the assumption that adding labels always improves integration. We found that methods relying heavily on labels, such as scDREAMER, excel with perfect or Azimuth-generated annotations but degrade rapidly with structured errors. Conversely, robust supervision methods like scANVI and ssSTACAS are stable but offer limited advantages. Based on these results, we recommend scDREAMER when annotations are high confidence, a state-of-the-art unsupervised method such as scCRAFT as the most reliable default when labels are incomplete or noisy, and scANVI as a robust semi-supervised alternative when partial labels are trusted.