Robustness of Deep ReLU Networks to Misclassification of High-Dimensional Data

TL;DR

This paper provides a theoretical analysis of how deep ReLU networks resist misclassification when subjected to small random input perturbations, especially in high-dimensional settings.

Contribution

It derives lower bounds on local robustness of deep ReLU networks based on input dimension and network size, advancing understanding of their stability.

Findings

Lower bounds on robustness depend on input dimensionality and network size

Deep ReLU networks exhibit quantifiable resilience to random perturbations

Theoretical insights into the robustness of high-dimensional neural networks

Abstract

We present a theoretical study of the robustness of parameterized networks to random input perturbations. Specifically, we analyze local robustness at a given network input by quantifying the probability that a small additive random perturbation of the input leads to misclassification. For deep networks with rectified linear units, we derive lower bounds on local robustness in terms of the input dimensionality and the total number of network units.