GCI-ViTAL: Gradual Confidence Improvement with Vision Transformers for Active Learning on Label Noise

TL;DR

This paper introduces GCI-ViTAL, a robust active learning algorithm using Vision Transformers that effectively handles label noise by selecting uncertain and semantically divergent samples, outperforming CNN-based methods across multiple datasets.

Contribution

The paper proposes GCI-ViTAL, a novel active learning method leveraging Vision Transformers and semantic information to improve robustness against label noise in image classification.

Findings

ViTs outperform CNNs in noisy label settings.

GCI-ViTAL effectively identifies uncertain and divergent samples.

Semantic information enhances model robustness under label noise.

Abstract

Active learning aims to train accurate classifiers while minimizing labeling costs by strategically selecting informative samples for annotation. This study focuses on image classification tasks, comparing AL methods on CIFAR10, CIFAR100, Food101, and the Chest X-ray datasets under varying label noise rates. We investigate the impact of model architecture by comparing Convolutional Neural Networks (CNNs) and Vision Transformer (ViT)-based models. Additionally, we propose a novel deep active learning algorithm, GCI-ViTAL, designed to be robust to label noise. GCI-ViTAL utilizes prediction entropy and the Frobenius norm of last-layer attention vectors compared to class-centric clean set attention vectors. Our method identifies samples that are both uncertain and semantically divergent from typical images in their assigned class. This allows GCI-ViTAL to select informative data points even in the presence of label noise while flagging potentially mislabeled candidates. Label smoothing is applied to train a model that is not overly confident about potentially noisy labels. We evaluate GCI-ViTAL under varying levels of symmetric label noise and compare it to five other AL strategies. Our results demonstrate that using ViTs leads to significant performance improvements over CNNs across all AL strategies, particularly in noisy label settings. We also find that using the semantic information of images as label grounding helps in training a more robust model under label noise. Notably, we do not perform extensive hyperparameter tuning, providing an out-of-the-box comparison that addresses the common challenge practitioners face in selecting models and active learning strategies without an exhaustive literature review on training and fine-tuning vision models on real-world application data.