Improved Mix-up with KL-Entropy for Learning From Noisy Labels

TL;DR

This paper introduces an improved joint optimization framework combining mix-up entropy and KL-entropy to enhance learning from noisy labels, demonstrating superior performance on benchmark datasets.

Contribution

The proposed method innovatively integrates mix-up and KL-entropy in the loss function for better handling noisy labels in deep neural network training.

Findings

Outperforms state-of-the-art methods on CIFAR-10.

Effective in fine-tuning with noisy annotations.

Shows robustness on Clothing1M dataset.

Abstract

Despite the deep neural networks (DNN) has achieved excellent performance in image classification researches, the training of DNNs needs a large of clean data with accurate annotations. The collect of a dataset is easy, but it is difficult to annotate the collecting data. On the websites, there exist a lot of image data which contains inaccurate annotations, but training on these datasets may make networks easier to over-fit the noisy labels and cause performance degradation. In this work, we propose an improved joint optimization framework, which mixed the mix-up entropy and Kullback-Leibler (KL) entropy as the loss function. The new loss function can give the better fine-tuning after the framework updates both the label annotations. We conduct experiments on CIFAR-10 dataset and Clothing1M dataset. The result shows the advantageous performance of our approach compared with other state-of-the-art methods.