NCIS: Deep Color Gradient Maps Regression and Three-Class Pixel Classification for Enhanced Neuronal Cell Instance Segmentation in Nissl-Stained Histological Images

TL;DR

This paper introduces an end-to-end deep learning framework that improves neuronal cell segmentation in Nissl-stained histological images by combining color gradient regression and pixel classification, enabling detailed cytoarchitecture analysis.

Contribution

It presents a novel U-Net-like architecture with dual decoding branches and attention gates for enhanced instance segmentation of neuronal cells in histological images.

Findings

Outperforms recent deep-learning methods in cell segmentation accuracy

Effective in cerebral cortex and cerebellum images

Enables detailed morphological analysis of brain tissue

Abstract

Deep learning has proven to be more effective than other methods in medical image analysis, including the seemingly simple but challenging task of segmenting individual cells, an essential step for many biological studies. Comparative neuroanatomy studies are an example where the instance segmentation of neuronal cells is crucial for cytoarchitecture characterization. This paper presents an end-to-end framework to automatically segment single neuronal cells in Nissl-stained histological images of the brain, thus aiming to enable solid morphological and structural analyses for the investigation of changes in the brain cytoarchitecture. A U-Net-like architecture with an EfficientNet as the encoder and two decoding branches is exploited to regress four color gradient maps and classify pixels into contours between touching cells, cell bodies, or background. The decoding branches are connected through attention gates to share relevant features, and their outputs are combined to return the instance segmentation of the cells. The method was tested on images of the cerebral cortex and cerebellum, outperforming other recent deep-learning-based approaches for the instance segmentation of cells.