Semantic Segmentation with Labeling Uncertainty and Class Imbalance

TL;DR

This paper introduces a novel pixel-weighting method that accounts for class imbalance and labeling uncertainty, significantly enhancing CNN-based semantic segmentation performance and robustness to noise.

Contribution

It proposes a new pixel-wise weighting approach that improves segmentation accuracy by addressing class imbalance and uncertainty during training.

Findings

Significant improvements over baseline methods in three segmentation tasks.

Enhanced robustness to noise in segmentation results.

Applicable to various semantic segmentation frameworks.

Abstract

Recently, methods based on Convolutional Neural Networks (CNN) achieved impressive success in semantic segmentation tasks. However, challenges such as the class imbalance and the uncertainty in the pixel-labeling process are not completely addressed. As such, we present a new approach that calculates a weight for each pixel considering its class and uncertainty during the labeling process. The pixel-wise weights are used during training to increase or decrease the importance of the pixels. Experimental results show that the proposed approach leads to significant improvements in three challenging segmentation tasks in comparison to baseline methods. It was also proved to be more invariant to noise. The approach presented here may be used within a wide range of semantic segmentation methods to improve their robustness.