Noise Sensitivity and Stability of Deep Neural Networks for Binary Classification

TL;DR

This paper investigates the noise sensitivity and stability of deep neural network classifiers for binary tasks by analyzing their Boolean function representations, revealing insights into their robustness properties.

Contribution

It introduces a Boolean function perspective to study DNN robustness, extending concepts to annealed and quenched versions, and analyzes standard architectures with Gaussian weights.

Findings

Relation between noise sensitivity and stability in DNNs clarified

Analysis of fully connected and convolutional models under Gaussian initialization

Insights into robustness phenomena of deep neural classifiers

Abstract

A first step is taken towards understanding often observed non-robustness phenomena of deep neural net (DNN) classifiers. This is done from the perspective of Boolean functions by asking if certain sequences of Boolean functions represented by common DNN models are noise sensitive or noise stable, concepts defined in the Boolean function literature. Due to the natural randomness in DNN models, these concepts are extended to annealed and quenched versions. Here we sort out the relation between these definitions and investigate the properties of two standard DNN architectures, the fully connected and convolutional models, when initiated with Gaussian weights.