InsMix: Towards Realistic Generative Data Augmentation for Nuclei Instance Segmentation

TL;DR

This paper introduces InsMix, a realistic data augmentation technique for nuclei segmentation that uses morphology constraints and background perturbation to improve deep learning model robustness and accuracy.

Contribution

InsMix is a novel augmentation method combining morphology-constrained generative instance augmentation with background perturbation and a smooth-GAN for contextual consistency.

Findings

Outperforms state-of-the-art methods on Kumar and CPS datasets.

Enhances model robustness and segmentation accuracy.

Effective use of morphology constraints and background shuffling.

Abstract

Nuclei Segmentation from histology images is a fundamental task in digital pathology analysis. However, deep-learning-based nuclei segmentation methods often suffer from limited annotations. This paper proposes a realistic data augmentation method for nuclei segmentation, named InsMix, that follows a Copy-Paste-Smooth principle and performs morphology-constrained generative instance augmentation. Specifically, we propose morphology constraints that enable the augmented images to acquire luxuriant information about nuclei while maintaining their morphology characteristics (e.g., geometry and location). To fully exploit the pixel redundancy of the background and improve the model's robustness, we further propose a background perturbation method, which randomly shuffles the background patches without disordering the original nuclei distribution. To achieve contextual consistency between original and template instances, a smooth-GAN is designed with a foreground similarity encoder (FSE) and a triplet loss. We validated the proposed method on two datasets, i.e., Kumar and CPS datasets. Experimental results demonstrate the effectiveness of each component and the superior performance achieved by our method to the state-of-the-art methods.