A Unified View of Piecewise Linear Neural Network Verification

TL;DR

This paper introduces a unified framework for verifying piecewise linear neural networks, significantly improving verification speed and providing a benchmark dataset for evaluating different algorithms.

Contribution

It presents a comprehensive framework that unifies existing verification methods, leading to faster algorithms and a new benchmark dataset for assessing verification techniques.

Findings

Achieved two orders of magnitude speedup over previous methods

Unified multiple verification approaches into a single framework

Provided the first comparative analysis of existing algorithms using a new benchmark

Abstract

The success of Deep Learning and its potential use in many safety-critical applications has motivated research on formal verification of Neural Network (NN) models. Despite the reputation of learned NN models to behave as black boxes and the theoretical hardness of proving their properties, researchers have been successful in verifying some classes of models by exploiting their piecewise linear structure and taking insights from formal methods such as Satisifiability Modulo Theory. These methods are however still far from scaling to realistic neural networks. To facilitate progress on this crucial area, we make two key contributions. First, we present a unified framework that encompasses previous methods. This analysis results in the identification of new methods that combine the strengths of multiple existing approaches, accomplishing a speedup of two orders of magnitude compared to the previous state of the art. Second, we propose a new data set of benchmarks which includes a collection of previously released testcases. We use the benchmark to provide the first experimental comparison of existing algorithms and identify the factors impacting the hardness of verification problems.