zoNNscan : a boundary-entropy index for zone inspection of neural models

TL;DR

zoNNscan is a novel boundary-entropy index that measures uncertainty around data points in neural network decision boundaries, aiding in identifying adversarial and corner case inputs.

Contribution

Introduces zoNNscan, a new index based on confidence entropy, for boundary inspection of neural models, with algorithms and applications demonstrated.

Findings

zoNNscan shows higher values for adversarial and corner case inputs

Effective in detecting boundary uncertainty in neural classifiers

Applicable to critical system safety assessments

Abstract

The training of deep neural network classifiers results in decision boundaries which geometry is still not well understood. This is in direct relation with classification problems such as so called adversarial examples. We introduce zoNNscan, an index that is intended to inform on the boundary uncertainty (in terms of the presence of other classes) around one given input datapoint. It is based on confidence entropy, and is implemented through sampling in the multidimensional ball surrounding that input. We detail the zoNNscan index, give an algorithm for approximating it, and finally illustrate its benefits on four applications, including two important problems for the adoption of deep networks in critical systems: adversarial examples and corner case inputs. We highlight that zoNNscan exhibits significantly higher values than for standard inputs in those two problem classes.