Learning to Detect Label Errors by Making Them: A Method for Segmentation and Object Detection Datasets

TL;DR

This paper introduces a unified, learning-based method for detecting label errors across multiple computer vision tasks, improving dataset quality and model performance by injecting and identifying label errors as an instance segmentation problem.

Contribution

A novel, generalizable approach for label error detection applicable to object detection, segmentation, and instance segmentation datasets, surpassing task-specific and non-learning methods.

Findings

Outperforms existing baselines and state-of-the-art methods in label error detection.

Effective on simulated label errors across diverse datasets and models.

Provides a new benchmark with 459 real label errors in Cityscapes.

Abstract

Recently, detection of label errors and improvement of label quality in datasets for supervised learning tasks has become an increasingly important goal in both research and industry. The consequences of incorrectly annotated data include reduced model performance, biased benchmark results, and lower overall accuracy. Current state-of-the-art label error detection methods often focus on a single computer vision task and, consequently, a specific type of dataset, containing, for example, either bounding boxes or pixel-wise annotations. Furthermore, previous methods are not learning-based. In this work, we overcome this research gap. We present a unified method for detecting label errors in object detection, semantic segmentation, and instance segmentation datasets. In a nutshell, our approach - learning to detect label errors by making them - works as follows: we inject different kinds of label errors into the ground truth. Then, the detection of label errors, across all mentioned primary tasks, is framed as an instance segmentation problem based on a composite input. In our experiments, we compare the label error detection performance of our method with various baselines and state-of-the-art approaches of each task's domain on simulated label errors across multiple tasks, datasets, and base models. This is complemented by a generalization study on real-world label errors. Additionally, we release 459 real label errors identified in the Cityscapes dataset and provide a benchmark for real label error detection in Cityscapes.