Adversarial VC-dimension and Sample Complexity of Neural Networks

TL;DR

This paper investigates the theoretical limits of neural network learning under adversarial attacks by analyzing the adversarial VC-dimension, providing insights into their sample complexity and robustness.

Contribution

It introduces the concept of adversarial VC-dimension for neural networks, linking network structure to adversarial robustness and sample complexity.

Findings

Derived the adversarial VC-dimension for neural networks with sign activation functions.

Established the relationship between the growth number of the network and individual neurons.

Provided theoretical bounds on the learnability of neural networks under adversarial conditions.

Abstract

Adversarial attacks during the testing phase of neural networks pose a challenge for the deployment of neural networks in security critical settings. These attacks can be performed by adding noise that is imperceptible to humans on top of the original data. By doing so, an attacker can create an adversarial sample, which will cause neural networks to misclassify. In this paper, we seek to understand the theoretical limits of what can be learned by neural networks in the presence of an adversary. We first defined the hypothesis space of a neural network, and showed the relationship between the growth number of the entire neural network and the growth number of each neuron. Combine that with the adversarial Vapnik-Chervonenkis(VC)-dimension of halfspace classifiers, we concluded the adversarial VC-dimension of the neural networks with sign activation functions.