Gaussian-Based and Outside-the-Box Runtime Monitoring Join Forces

TL;DR

This paper introduces a novel runtime monitoring approach for neural networks by combining Gaussian-based and Outside-the-Box methods, enhancing the detection of abnormal activations in safety-critical applications.

Contribution

It proposes a new hybrid monitoring technique that integrates Gaussian-based and cluster-based methods to improve neural network behavior monitoring.

Findings

Improved detection of abnormal neuron activations.

Enhanced monitoring accuracy in safety-critical scenarios.

Demonstrated effectiveness through experimental evaluation.

Abstract

Since neural networks can make wrong predictions even with high confidence, monitoring their behavior at runtime is important, especially in safety-critical domains like autonomous driving. In this paper, we combine ideas from previous monitoring approaches based on observing the activation values of hidden neurons. In particular, we combine the Gaussian-based approach, which observes whether the current value of each monitored neuron is similar to typical values observed during training, and the Outside-the-Box monitor, which creates clusters of the acceptable activation values, and, thus, considers the correlations of the neurons' values. Our experiments evaluate the achieved improvement.