ROIsGAN: A Region Guided Generative Adversarial Framework for Murine Hippocampal Subregion Segmentation

TL;DR

This paper introduces ROIsGAN, a novel region-guided GAN framework for automated segmentation of murine hippocampal subregions from histological images, supported by new comprehensive datasets and demonstrating superior performance over existing models.

Contribution

The paper presents a new GAN-based segmentation model tailored for hippocampal subregions and provides publicly available datasets, advancing automated analysis in neuroscience.

Findings

ROIsGAN outperforms conventional models with 1-10% higher Dice scores.

Achieves up to 11% improvement in IoU under challenging staining conditions.

Provides publicly available datasets and code for further research.

Abstract

The hippocampus, a critical brain structure involved in memory processing and various neurodegenerative and psychiatric disorders, comprises three key subregions: the dentate gyrus (DG), Cornu Ammonis 1 (CA1), and Cornu Ammonis 3 (CA3). Accurate segmentation of these subregions from histological tissue images is essential for advancing our understanding of disease mechanisms, developmental dynamics, and therapeutic interventions. However, no existing methods address the automated segmentation of hippocampal subregions from tissue images, particularly from immunohistochemistry (IHC) images. To bridge this gap, we introduce a novel set of four comprehensive murine hippocampal IHC datasets featuring distinct staining modalities: cFos, NeuN, and multiplexed stains combining cFos, NeuN, and either {\Delta}FosB or GAD67, capturing structural, neuronal activity, and plasticity associated information. Additionally, we propose ROIsGAN, a region-guided U-Net-based generative adversarial network tailored for hippocampal subregion segmentation. By leveraging adversarial learning, ROIsGAN enhances boundary delineation and structural detail refinement through a novel region-guided discriminator loss combining Dice and binary cross-entropy loss. Evaluated across DG, CA1, and CA3 subregions, ROIsGAN consistently outperforms conventional segmentation models, achieving performance gains ranging from 1-10% in Dice score and up to 11% in Intersection over Union (IoU), particularly under challenging staining conditions. Our work establishes foundational datasets and methods for automated hippocampal segmentation, enabling scalable, high-precision analysis of tissue images in neuroscience research. Our generated datasets, proposed model as a standalone tool, and its corresponding source code are publicly available at: https://github.com/MehediAzim/ROIsGAN