# RNA2seg: a generalist model for cell segmentation in image-based spatial transcriptomics

## TL;DR

RNA2seg is a new cell segmentation tool for spatial transcriptomics that improves accuracy by using RNA data and cell stains.

## Contribution

RNA2seg introduces a generalist segmentation model trained on diverse datasets using a teacher-student approach.

## Key findings

- RNA2seg shows superior performance in cell segmentation using RNA point clouds and stainings.
- The model performs well in zero-shot and few-shot settings.
- It was trained on over 4 million cells from MERFISH and CosMx datasets across seven organs.

## Abstract

Imaging-based spatial transcriptomics enables high-resolution spatial mapping of RNA species. A key challenge in imaging-based spatial transcriptomics is accurate cell segmentation to assign each RNA molecule to the right cell. Here, we present RNA2seg, a novel segmentation algorithm trained on over 4 million cells from MERFISH and CosMx datasets across seven organs using a teacher-student training scheme. RNA2seg integrates RNA point clouds and all available membrane and nuclear stainings. Validation on manually annotated data shows superior performance including in zero-shot and few-shot settings.

The online version contains supplementary material available at 10.1186/s13059-025-03908-9.