Classifier-Guided Visual Correction of Noisy Labels for Image Classification Tasks

TL;DR

This paper introduces a classifier-guided visual correction method for noisy labels in image classification, enabling users to interactively identify and fix labeling errors to improve training data quality.

Contribution

It proposes a novel interactive approach using pretrained classifiers to detect, reason about, and correct labeling errors, with a systematic categorization of error types and visual guidance techniques.

Findings

Effective error detection in benchmark datasets

Improved data quality through user interaction

Techniques applicable beyond images

Abstract

Training data plays an essential role in modern applications of machine learning. However, gathering labeled training data is time-consuming. Therefore, labeling is often outsourced to less experienced users, or completely automated. This can introduce errors, which compromise valuable training data, and lead to suboptimal training results. We thus propose a novel approach that uses the power of pretrained classifiers to visually guide users to noisy labels, and let them interactively check error candidates, to iteratively improve the training data set. To systematically investigate training data, we propose a categorization of labeling errors into three different types, based on an analysis of potential pitfalls in label acquisition processes. For each of these types, we present approaches to detect, reason about, and resolve error candidates, as we propose measures and visual guidance techniques to support machine learning users. Our approach has been used to spot errors in well-known machine learning benchmark data sets, and we tested its usability during a user evaluation. While initially developed for images, the techniques presented in this paper are independent of the classification algorithm, and can also be extended to many other types of training data.