Adaptive Multi-Scale Integration Unlocks Robust Cell Annotation in Histopathology Images

TL;DR

This paper introduces NuClass, a multi-scale, uncertainty-aware framework that integrates nuclear morphology and tissue context to improve cell annotation accuracy in histopathology images, achieving up to 96% F1 score.

Contribution

The study presents a novel multi-scale, marker-guided approach combining local and global tissue information with uncertainty modeling for robust cell classification.

Findings

Achieves up to 96% F1 score on cell classification.

Outperforms existing baselines in histopathology cell annotation.

Effectively integrates nuclear and microenvironmental features.

Abstract

Identifying cell types and subtypes in routine histopathology is fundamental for understanding disease. Existing tile-based models capture nuclear detail but miss the broader tissue context that influences cell identity. Current human annotations are coarse-grained and uneven across studies, making fine-grained, subtype-level classification difficult. In this study, we build a marker-guided dataset from Xenium spatial transcriptomics with single-cell resolution labels for more than two million cells across eight organs and 16 classes to address the lack of high-quality annotations. Leveraging this data resource, we introduce NuClass, a pathologist workflow inspired framework for cell-wise multi-scale integration of nuclear morphology and microenvironmental context. It combines Path local, which focuses on nuclear morphology from 224x224 pixel crops, and Path global, which models the surrounding 1024x1024 pixel neighborhood, through a learnable gating module that balances local and global information. An uncertainty-guided objective directs the global path to prioritize regions where the local path is uncertain, and we provide calibrated confidence estimates and Grad-CAM maps for interpretability. Evaluated on three fully held-out cohorts, NuClass achieves up to 96 percent F1 for its best-performing class, outperforming strong baselines. Our results demonstrate that multi-scale, uncertainty-aware fusion can bridge the gap between slide-level pathological foundation models and reliable, cell-level phenotype prediction.