Domain Adaptive and Fine-grained Anomaly Detection for Single-cell Sequencing Data and Beyond

TL;DR

This paper introduces ACSleuth, a new generative framework that improves anomaly detection and subtyping in single-cell sequencing data across multiple domains, addressing domain shifts and providing theoretical insights.

Contribution

The paper presents the first theoretical analysis of reconstruction deviations for anomaly detection and develops a novel MMD-based scoring method within ACSleuth.

Findings

ACSleuth outperforms state-of-the-art methods in anomaly detection and subtyping.

Theoretical analysis supports the use of reconstruction deviations for domain shift handling.

Extensive benchmarks demonstrate ACSleuth's effectiveness across various datasets.

Abstract

Fined-grained anomalous cell detection from affected tissues is critical for clinical diagnosis and pathological research. Single-cell sequencing data provide unprecedented opportunities for this task. However, current anomaly detection methods struggle to handle domain shifts prevalent in multi-sample and multi-domain single-cell sequencing data, leading to suboptimal performance. Moreover, these methods fall short of distinguishing anomalous cells into pathologically distinct subtypes. In response, we propose ACSleuth, a novel, reconstruction deviation-guided generative framework that integrates the detection, domain adaptation, and fine-grained annotating of anomalous cells into a methodologically cohesive workflow. Notably, we present the first theoretical analysis of using reconstruction deviations output by generative models for anomaly detection in lieu of domain shifts. This analysis informs us to develop a novel and superior maximum mean discrepancy-based anomaly scorer in ACSleuth. Extensive benchmarks over various single-cell data and other types of tabular data demonstrate ACSleuth's superiority over the state-of-the-art methods in identifying and subtyping anomalies in multi-sample and multi-domain contexts. Our code is available at https://github.com/Catchxu/ACsleuth.