Channel Boosted CNN-Transformer-based Multi-Level and Multi-Scale Nuclei Segmentation

TL;DR

This paper introduces two novel CNN-Transformer hybrid architectures, NucleiHVT and CB-NucleiHVT, that improve nuclei segmentation accuracy in histology images by capturing multi-scale features and enhancing feature diversity.

Contribution

The paper proposes two innovative CNN-Transformer based models that combine dual attention and channel boosting to improve nuclei segmentation in histology images, outperforming existing methods.

Findings

Outperform existing CNN, Transformer, and hybrid methods in segmentation accuracy.

Effective in capturing multi-scale and diverse features for nuclei delineation.

Showed superior results on multiple medical image datasets.

Abstract

Accurate nuclei segmentation is an essential foundation for various applications in computational pathology, including cancer diagnosis and treatment planning. Even slight variations in nuclei representations can significantly impact these downstream tasks. However, achieving accurate segmentation remains challenging due to factors like clustered nuclei, high intra-class variability in size and shape, resemblance to other cells, and color or contrast variations between nuclei and background. Despite the extensive utilization of Convolutional Neural Networks (CNNs) in medical image segmentation, they may have trouble capturing long-range dependencies crucial for accurate nuclei delineation. Transformers address this limitation but might miss essential low-level features. To overcome these limitations, we utilized CNN-Transformer-based techniques for nuclei segmentation in H&E stained histology images. In this work, we proposed two CNN-Transformer architectures, Nuclei Hybrid Vision Transformer (NucleiHVT) and Channel Boosted Nuclei Hybrid Vision Transformer (CB-NucleiHVT), that leverage the strengths of both CNNs and Transformers to effectively learn nuclei boundaries in multi-organ histology images. The first architecture, NucleiHVT is inspired by the UNet architecture and incorporates the dual attention mechanism to capture both multi-level and multi-scale context effectively. The CB-NucleiHVT network, on the other hand, utilizes the concept of channel boosting to learn diverse feature spaces, enhancing the model's ability to distinguish subtle variations in nuclei characteristics. Detailed evaluation of two medical image segmentation datasets shows that the proposed architectures outperform existing CNN-based, Transformer-based, and hybrid methods. The proposed networks demonstrated effective results both in terms of quantitative metrics, and qualitative visual assessment.