Sparse Polynomial Optimisation for Neural Network Verification

TL;DR

This paper introduces a novel approach for neural network verification using sparse polynomial optimisation and the Positivstellensatz, achieving tighter bounds with reasonable computation times compared to existing methods.

Contribution

It applies sparse polynomial optimisation and real algebraic geometry to neural network verification, improving bound tightness and computational efficiency over traditional bounding methods.

Findings

Tighter bounds achieved with sparse polynomial optimisation.

Computational time remains reasonable despite increased accuracy.

Outperforms existing methods for networks with ReLU, sigmoid, and tanh activations.

Abstract

The prevalence of neural networks in society is expanding at an increasing rate. It is becoming clear that providing robust guarantees on systems that use neural networks is very important, especially in safety-critical applications. A trained neural network's sensitivity to adversarial attacks is one of its greatest shortcomings. To provide robust guarantees, one popular method that has seen success is to bound the activation functions using equality and inequality constraints. However, there are numerous ways to form these bounds, providing a trade-off between conservativeness and complexity. Depending on the complexity of these bounds, the computational time of the optimisation problem varies, with longer solve times often leading to tighter bounds. We approach the problem from a different perspective, using sparse polynomial optimisation theory and the Positivstellensatz, which derives from the field of real algebraic geometry. The former exploits the natural cascading structure of the neural network using ideas from chordal sparsity while the later asserts the emptiness of a semi-algebraic set with a nested family of tests of non-decreasing accuracy to provide tight bounds. We show that bounds can be tightened significantly, whilst the computational time remains reasonable. We compare the solve times of different solvers and show how the accuracy can be improved at the expense of increased computation time. We show that by using this sparse polynomial framework the solve time and accuracy can be improved over other methods for neural network verification with ReLU, sigmoid and tanh activation functions.