Towards the Quantification of Safety Risks in Deep Neural Networks

TL;DR

This paper introduces a general framework for quantifying safety risks in deep neural networks by measuring the maximum safe norm radius, encompassing various risk types including a newly identified uncertainty risk, with an efficient GPU-accelerated algorithm.

Contribution

It proposes a generic safety risk quantification method applicable to diverse risks and neural network structures, introducing a new uncertainty risk class and an efficient computation algorithm.

Findings

Method achieves competitive tightness and efficiency in safety quantification.

Supports broad class of safety risks without network restrictions.

Successfully evaluated on multiple benchmark neural networks.

Abstract

Safety concerns on the deep neural networks (DNNs) have been raised when they are applied to critical sectors. In this paper, we define safety risks by requesting the alignment of the network's decision with human perception. To enable a general methodology for quantifying safety risks, we define a generic safety property and instantiate it to express various safety risks. For the quantification of risks, we take the maximum radius of safe norm balls, in which no safety risk exists. The computation of the maximum safe radius is reduced to the computation of their respective Lipschitz metrics - the quantities to be computed. In addition to the known adversarial example, reachability example, and invariant example, in this paper we identify a new class of risk - uncertainty example - on which humans can tell easily but the network is unsure. We develop an algorithm, inspired by derivative-free optimization techniques and accelerated by tensor-based parallelization on GPUs, to support efficient computation of the metrics. We perform evaluations on several benchmark neural networks, including ACSC-Xu, MNIST, CIFAR-10, and ImageNet networks. The experiments show that, our method can achieve competitive performance on safety quantification in terms of the tightness and the efficiency of computation. Importantly, as a generic approach, our method can work with a broad class of safety risks and without restrictions on the structure of neural networks.