Expediting Neural Network Verification via Network Reduction

TL;DR

This paper introduces a network reduction technique that simplifies neural networks by removing stable ReLU neurons, enabling faster and more effective verification with existing tools.

Contribution

It proposes a novel pre-processing method that reduces network complexity, improving verification efficiency and applicability for large and complicated neural networks.

Findings

Significantly reduces neural network size.

Speeds up existing verification tools.

Enhances verification success rate on complex networks.

Abstract

A wide range of verification methods have been proposed to verify the safety properties of deep neural networks ensuring that the networks function correctly in critical applications. However, many well-known verification tools still struggle with complicated network architectures and large network sizes. In this work, we propose a network reduction technique as a pre-processing method prior to verification. The proposed method reduces neural networks via eliminating stable ReLU neurons, and transforming them into a sequential neural network consisting of ReLU and Affine layers which can be handled by the most verification tools. We instantiate the reduction technique on the state-of-the-art complete and incomplete verification tools, including alpha-beta-crown, VeriNet and PRIMA. Our experiments on a large set of benchmarks indicate that the proposed technique can significantly reduce neural networks and speed up existing verification tools. Furthermore, the experiment results also show that network reduction can improve the availability of existing verification tools on many networks by reducing them into sequential neural networks.