Uni-Hema: Unified Model for Digital Hematopathology

TL;DR

Uni-Hema is a comprehensive multi-task, multi-modal model for digital hematopathology that integrates various analysis tasks across multiple diseases, leveraging extensive datasets to improve performance and interpretability.

Contribution

The paper introduces Uni-Hema, a novel unified model that combines detection, classification, segmentation, and reasoning in digital hematopathology, surpassing single-task models in performance.

Findings

Achieves comparable or superior performance to single-task models.

Provides interpretable, morphologically relevant insights at the single-cell level.

Utilizes 46 datasets with over 700K images and 21K QA pairs.

Abstract

Digital hematopathology requires cell-level analysis across diverse disease categories, including malignant disorders (e.g., leukemia), infectious conditions (e.g., malaria), and non-malignant red blood cell disorders (e.g., sickle cell disease). Whether single-task, vision-language, WSI-optimized, or single-cell hematology models, these approaches share a key limitation, they cannot provide unified, multi-task, multi-modal reasoning across the complexities of digital hematopathology. To overcome these limitations, we propose Uni-Hema, a multi-task, unified model for digital hematopathology integrating detection, classification, segmentation, morphology prediction, and reasoning across multiple diseases. Uni-Hema leverages 46 publicly available datasets, encompassing over 700K images and 21K question-answer pairs, and is built upon Hema-Former, a multimodal module that bridges visual and textual representations at the hierarchy level for the different tasks (detection, classification, segmentation, morphology, mask language modeling and visual question answer) at different granularity. Extensive experiments demonstrate that Uni-Hema achieves comparable or superior performance to train on a single-task and single dataset models, across diverse hematological tasks, while providing interpretable, morphologically relevant insights at the single-cell level. Our framework establishes a new standard for multi-task and multi-modal digital hematopathology. The code will be made publicly available.