Neural Network Robustness as a Verification Property: A Principled Case Study

TL;DR

This paper proposes a systematic framework for analyzing neural network robustness as a formal verification property, addressing the lack of comparative analysis among existing notions of robustness through a case study.

Contribution

It introduces general principles for empirical evaluation of neural network robustness as a mathematical property during training, verification, and deployment.

Findings

Framework enables systematic comparison of robustness notions

Case study demonstrates practical benefits of the approach

Highlights the importance of formal verification in robustness assessment

Abstract

Neural networks are very successful at detecting patterns in noisy data, and have become the technology of choice in many fields. However, their usefulness is hampered by their susceptibility to adversarial attacks. Recently, many methods for measuring and improving a network's robustness to adversarial perturbations have been proposed, and this growing body of research has given rise to numerous explicit or implicit notions of robustness. Connections between these notions are often subtle, and a systematic comparison between them is missing in the literature. In this paper we begin addressing this gap, by setting up general principles for the empirical analysis and evaluation of a network's robustness as a mathematical property - during the network's training phase, its verification, and after its deployment. We then apply these principles and conduct a case study that showcases the practical benefits of our general approach.