Exact Upper and Lower Bounds for the Output Distribution of Neural Networks with Random Inputs

TL;DR

This paper develops a method to compute exact upper and lower bounds for the output distribution of neural networks with noisy inputs, applicable to various architectures and activation functions, providing guaranteed error bounds.

Contribution

It introduces a novel approach to bound neural network output distributions using ReLU neural networks, applicable to a wide range of architectures and activation functions.

Findings

Provides guaranteed bounds for neural network output distributions.

Applicable to feedforward and convolutional neural networks.

Demonstrates accurate bounds with experimental validation.

Abstract

We derive exact upper and lower bounds for the cumulative distribution function (cdf) of the output of a neural network (NN) over its entire support subject to noisy (stochastic) inputs. The upper and lower bounds converge to the true cdf over its domain as the resolution increases. Our method applies to any feedforward NN using continuous monotonic piecewise twice continuously differentiable activation functions (e.g., ReLU, tanh and softmax) and convolutional NNs, which were beyond the scope of competing approaches. The novelty and instrumental tool of our approach is to bound general NNs with ReLU NNs. The ReLU NN-based bounds are then used to derive the upper and lower bounds of the cdf of the NN output. Experiments demonstrate that our method delivers guaranteed bounds of the predictive output distribution over its support, thus providing exact error guarantees, in contrast to competing approaches.