Noisy Annotations in Semantic Segmentation

TL;DR

This paper investigates the impact of noisy annotations on semantic segmentation, introduces realistic noise benchmarks, and evaluates model robustness and generalization under these conditions.

Contribution

It introduces COCO-N, Cityscapes-N, and COCO-WAN benchmarks for assessing segmentation model robustness to annotation noise, a novel approach in the field.

Findings

Models' performance degrades with increased noise levels.

Benchmark datasets reveal limitations of existing noise-robust methods.

Foundation models can simulate semi-automated annotation noise.

Abstract

Obtaining accurate labels for instance segmentation is particularly challenging due to the complex nature of the task. Each image necessitates multiple annotations, encompassing not only the object class but also its precise spatial boundaries. These requirements elevate the likelihood of errors and inconsistencies in both manual and automated annotation processes. By simulating different noise conditions, we provide a realistic scenario for assessing the robustness and generalization capabilities of instance segmentation models in different segmentation tasks, introducing COCO-N and Cityscapes-N. We also propose a benchmark for weakly annotation noise, dubbed COCO-WAN, which utilizes foundation models and weak annotations to simulate semi-automated annotation tools and their noisy labels. This study sheds light on the quality of segmentation masks produced by various models and challenges the efficacy of popular methods designed to address learning with label noise.