Novelty Detection Through Model-Based Characterization of Neural Networks

TL;DR

This paper introduces a model-based approach for novelty detection in neural networks using backpropagated gradients, outperforming activation-based methods across multiple image datasets.

Contribution

It demonstrates that gradients provide a more effective characterization of abnormal inputs than activation-based representations in neural networks.

Findings

Gradients outperform activations in novelty detection accuracy.

Achieved high AUROC scores on four image datasets.

Validated the effectiveness of model-based characterization for abnormal input detection.

Abstract

In this paper, we propose a model-based characterization of neural networks to detect novel input types and conditions. Novelty detection is crucial to identify abnormal inputs that can significantly degrade the performance of machine learning algorithms. Majority of existing studies have focused on activation-based representations to detect abnormal inputs, which limits the characterization of abnormality from a data perspective. However, a model perspective can also be informative in terms of the novelties and abnormalities. To articulate the significance of the model perspective in novelty detection, we utilize backpropagated gradients. We conduct a comprehensive analysis to compare the representation capability of gradients with that of activation and show that the gradients outperform the activation in novel class and condition detection. We validate our approach using four image recognition datasets including MNIST, Fashion-MNIST, CIFAR-10, and CURE-TSR. We achieve a significant improvement on all four datasets with an average AUROC of 0.953, 0.918, 0.582, and 0.746, respectively.