# Pattern Learning and Knowledge Distillation for Single-Cell Data Annotation

## TL;DR

This paper introduces PLKD, a new method for improving cell type annotation in single-cell data by using pattern learning and knowledge distillation to reduce batch effects and enhance accuracy.

## Contribution

The novel contribution is PLKD, which combines pattern-level biological insights with knowledge distillation for efficient and accurate cell type annotation.

## Key findings

- PLKD achieves high accuracy and robustness in cell type annotation while reducing batch effects.
- The method supports multi-modal cell type annotation and integration tasks efficiently.
- Knowledge distillation enables a lightweight model that is resistant to noise and fast inference.

## Abstract

Single-cell technologies allow researchers to measure gene expression at the level of individual cells, providing a powerful way to study cell identities and understand biological processes. However, differences between datasets—known as batch effects—can make accurate cell type annotation challenging. The existing deep learning methods often require heavy computation or fail to jointly address batch correction and cell type identification. In this study, we introduce PLKD, a new method that uses biologically meaningful gene groups, called patterns, together with knowledge distillation to improve cell type annotation. PLKD first learns pattern-level information using a Transformer-based Teacher model and then transfers this knowledge to a lightweight Student model. This design enables PLKD to accurately classify cell types, reduce batch effects, and provide interpretable biological insights. Our results show that PLKD achieves high accuracy and robustness while remaining efficient for large-scale datasets, offering a practical and interpretable tool for single-cell analysis.

Transferring cell type annotations from reference dataset to query dataset is a fundamental problem in AI-based single-cell data analysis. However, single-cell measurement techniques lead to domain gaps between multiple batches or datasets. The existing deep learning methods lack consideration on batch integration when learning reference annotations, which is a challenge for cell type annotation on multiple query batches. For cell representation, batch integration can not only eliminate the gaps between batches or datasets but also improve the heterogeneity of cell clusters. In this study, we proposed PLKD, a cell type annotation method based on pattern learning and knowledge distillation. PLKD consists of Teacher (Transformer) and Student (MLP). Teacher groups all input genes (features) into different gene sets (patterns), and each pattern represents a specific biological function. This design enables model to focus on biologically relevant functions interaction rather than gene-level expression that is susceptible to gaps of batches. In addition, knowledge distillation makes lightweight Student resistant to noise, allowing Student to infer quickly and robustly. Furthermore, PLKD supports multi-modal cell type annotation, multi-modal integration and other tasks. Benchmark experiments demonstrate that PLKD is able to achieve accurate and robust cell type annotation.