Inconsistency Ranking-based Noisy Label Detection for High-quality Data

TL;DR

This paper introduces an automatic noisy label detection method based on inconsistency ranking, aimed at improving data quality for deep learning, demonstrated on speaker verification tasks with promising results.

Contribution

It proposes a novel inconsistency ranking approach for noisy label detection, enhancing data cleaning efficiency and effectiveness in large-scale datasets.

Findings

Improves noisy label detection accuracy in speaker verification datasets.

Outperforms baseline methods in cleaning efficiency and effectiveness.

Effective across various noise levels and metric learning loss functions.

Abstract

The success of deep learning requires high-quality annotated and massive data. However, the size and the quality of a dataset are usually a trade-off in practice, as data collection and cleaning are expensive and time-consuming. In real-world applications, especially those using crowdsourcing datasets, it is important to exclude noisy labels. To address this, this paper proposes an automatic noisy label detection (NLD) technique with inconsistency ranking for high-quality data. We apply this technique to the automatic speaker verification (ASV) task as a proof of concept. We investigate both inter-class and intra-class inconsistency ranking and compare several metric learning loss functions under different noise settings. Experimental results confirm that the proposed solution could increase both the efficient and effective cleaning of large-scale speaker recognition datasets.