Neural Network Branching for Neural Network Verification

TL;DR

This paper introduces a graph neural network-based framework for neural network verification, significantly improving the efficiency of branch-and-bound methods by learning effective branching strategies.

Contribution

It presents a novel GNN-based approach that directly models neural networks as graphs for verification, outperforming traditional heuristics and demonstrating transferability across network sizes and property difficulties.

Findings

50% reduction in branches and verification time

Effective transferability across network sizes and property complexities

Outperforms existing hand-designed branching strategies

Abstract

Formal verification of neural networks is essential for their deployment in safety-critical areas. Many available formal verification methods have been shown to be instances of a unified Branch and Bound (BaB) formulation. We propose a novel framework for designing an effective branching strategy for BaB. Specifically, we learn a graph neural network (GNN) to imitate the strong branching heuristic behaviour. Our framework differs from previous methods for learning to branch in two main aspects. Firstly, our framework directly treats the neural network we want to verify as a graph input for the GNN. Secondly, we develop an intuitive forward and backward embedding update schedule. Empirically, our framework achieves roughly $50\%$ reduction in both the number of branches and the time required for verification on various convolutional networks when compared to the best available hand-designed branching strategy. In addition, we show that our GNN model enjoys both horizontal and vertical transferability. Horizontally, the model trained on easy properties performs well on properties of increased difficulty levels. Vertically, the model trained on small neural networks achieves similar performance on large neural networks.