Automated Cleanup of the ImageNet Dataset by Model Consensus, Explainability and Confident Learning

TL;DR

This paper presents automated methods using model consensus, explainability, and confident learning to identify and correct labeling errors in ImageNet, improving model performance and emphasizing the importance of dataset quality.

Contribution

It introduces automated heuristics for dataset cleaning based on model consensus, explainability, and confident learning, and demonstrates their effectiveness on ImageNet.

Findings

ImageNet-Clean improves model accuracy by 2-2.4%.

Fixing dataset errors reduces bias and improves transfer learning.

Widescreen input resolutions impact training performance.

Abstract

The convolutional neural networks (CNNs) trained on ILSVRC12 ImageNet were the backbone of various applications as a generic classifier, a feature extractor or a base model for transfer learning. This paper describes automated heuristics based on model consensus, explainability and confident learning to correct labeling mistakes and remove ambiguous images from this dataset. After making these changes on the training and validation sets, the ImageNet-Clean improves the model performance by 2-2.4 % for SqueezeNet and EfficientNet-B0 models. The results support the importance of larger image corpora and semi-supervised learning, but the original datasets must be fixed to avoid transmitting their mistakes and biases to the student learner. Further contributions describe the training impacts of widescreen input resolutions in portrait and landscape orientations. The trained models and scripts are published on Github (https://github.com/kecsap/imagenet-clean) to clean up ImageNet and ImageNetV2 datasets for reproducible research.