Adversarial Stain Transfer to Study the Effect of Color Variation on Cell Instance Segmentation

TL;DR

This study investigates how color variation in histological images affects cell segmentation performance, demonstrating the importance of stain normalization through a deep learning-based color transfer approach.

Contribution

It introduces a deep learning method for stain color transfer and quantitatively assesses the impact of color variation on cell segmentation accuracy.

Findings

Color variation significantly impacts segmentation results.

Stain normalization improves segmentation consistency.

Deep learning-based color transfer effectively simulates stain variations.

Abstract

Stain color variation in histological images, caused by a variety of factors, is a challenge not only for the visual diagnosis of pathologists but also for cell segmentation algorithms. To eliminate the color variation, many stain normalization approaches have been proposed. However, most were designed for hematoxylin and eosin staining images and performed poorly on immunohistochemical staining images. Current cell segmentation methods systematically apply stain normalization as a preprocessing step, but the impact brought by color variation has not been quantitatively investigated yet. In this paper, we produced five groups of NeuN staining images with different colors. We applied a deep learning image-recoloring method to perform color transfer between histological image groups. Finally, we altered the color of a segmentation set and quantified the impact of color variation on cell segmentation. The results demonstrated the necessity of color normalization prior to subsequent analysis.