The Exploration of Error Bounds in Classification with Noisy Labels

TL;DR

This paper analyzes the theoretical error bounds for classifiers trained with deep neural networks on noisy labels, addressing statistical and approximation errors, and extends results to dependent data and low-dimensional manifolds.

Contribution

It derives novel error bounds for excess risk in noisy label classification, incorporating dependencies and high-dimensional considerations within deep learning frameworks.

Findings

Derived error bounds for excess risk with noisy labels.

Extended bounds to dependent data using block construction.

Refined approximation error under low-dimensional manifold hypothesis.

Abstract

Numerous studies have shown that label noise can lead to poor generalization performance, negatively affecting classification accuracy. Therefore, understanding the effectiveness of classifiers trained using deep neural networks in the presence of noisy labels is of considerable practical significance. In this paper, we focus on the error bounds of excess risks for classification problems with noisy labels within deep learning frameworks. We derive error bounds for the excess risk, decomposing it into statistical error and approximation error. To handle statistical dependencies (e.g., mixing sequences), we employ an independent block construction to bound the error, leveraging techniques for dependent processes. For the approximation error, we establish these theoretical results to the vector-valued setting, where the output space consists of $K$-dimensional unit vectors. Finally, under the low-dimensional manifold hypothesis, we further refine the approximation error to mitigate the impact of high-dimensional input spaces.