Robustness of classifiers: from adversarial to random noise

TL;DR

This paper introduces a semi-random noise regime to analyze the robustness of classifiers, providing theoretical bounds that interpolate between adversarial and random noise, supported by experiments on neural networks.

Contribution

It offers the first quantitative analysis of classifier robustness in a semi-random noise regime, linking boundary curvature to noise resilience.

Findings

Robustness bounds depend on decision boundary curvature.

Bounds accurately predict neural network robustness.

Interpolate between worst-case and random noise regimes.

Abstract

Several recent works have shown that state-of-the-art classifiers are vulnerable to worst-case (i.e., adversarial) perturbations of the datapoints. On the other hand, it has been empirically observed that these same classifiers are relatively robust to random noise. In this paper, we propose to study a \textit{semi-random} noise regime that generalizes both the random and worst-case noise regimes. We propose the first quantitative analysis of the robustness of nonlinear classifiers in this general noise regime. We establish precise theoretical bounds on the robustness of classifiers in this general regime, which depend on the curvature of the classifier's decision boundary. Our bounds confirm and quantify the empirical observations that classifiers satisfying curvature constraints are robust to random noise. Moreover, we quantify the robustness of classifiers in terms of the subspace dimension in the semi-random noise regime, and show that our bounds remarkably interpolate between the worst-case and random noise regimes. We perform experiments and show that the derived bounds provide very accurate estimates when applied to various state-of-the-art deep neural networks and datasets. This result suggests bounds on the curvature of the classifiers' decision boundaries that we support experimentally, and more generally offers important insights onto the geometry of high dimensional classification problems.