Towards High-Quality Image Segmentation: Improving Topology Accuracy by Penalizing Neighbor Pixels

TL;DR

This paper introduces SCNP, a novel and efficient method to enhance topology accuracy in image segmentation by penalizing neighbor pixels, applicable across various datasets and frameworks, improving reliability of segmentation results.

Contribution

The paper proposes SCNP, a new neighbor-based penalization technique that improves topology accuracy in segmentation models, easily integrable into existing frameworks and loss functions.

Findings

SCNP improves topology accuracy across 13 diverse datasets.

SCNP can be integrated into multiple segmentation frameworks.

SCNP enhances various loss functions to better preserve structures.

Abstract

Standard deep learning models for image segmentation cannot guarantee topology accuracy, failing to preserve the correct number of connected components or structures. This, in turn, affects the quality of the segmentations and compromises the reliability of the subsequent quantification analyses. Previous works have proposed to enhance topology accuracy with specialized frameworks, architectures, and loss functions. However, these methods are often cumbersome to integrate into existing training pipelines, they are computationally very expensive, or they are restricted to structures with tubular morphology. We present SCNP, an efficient method that improves topology accuracy by penalizing the logits with their poorest-classified neighbor, forcing the model to improve the prediction at the pixels' neighbors before allowing it to improve the pixels themselves. We show the effectiveness of SCNP across 13 datasets, covering different structure morphologies and image modalities, and integrate it into three frameworks for semantic and instance segmentation. Additionally, we show that SCNP can be integrated into several loss functions, making them improve topology accuracy. Our code can be found at https://jmlipman.github.io/SCNP-SameClassNeighborPenalization.