SAVER: A Toolbox for Sampling-Based, Probabilistic Verification of Neural Networks

TL;DR

SAVER is a toolbox that uses sampling-based probabilistic methods to verify and modify neural network outputs to meet specified satisfaction probabilities with confidence.

Contribution

It introduces a novel toolbox for probabilistic neural network verification and set expansion, combining sampling and signed distance functions for improved assessment.

Findings

Successfully assesses likelihood of neural network outputs within sets.

Provides methods to modify constraints to meet satisfaction probabilities.

Offers a probabilistic approach with confidence guarantees.

Abstract

We present a neural network verification toolbox to 1) assess the probability of satisfaction of a constraint, and 2) synthesize a set expansion factor to achieve the probability of satisfaction. Specifically, the tool box establishes with a user-specified level of confidence whether the output of the neural network for a given input distribution is likely to be contained within a given set. Should the tool determine that the given set cannot satisfy the likelihood constraint, the tool also implements an approach outlined in this paper to alter the constraint set to ensure that the user-defined satisfaction probability is achieved. The toolbox is comprised of sampling-based approaches which exploit the properties of signed distance function to define set containment.