Statistical process monitoring of artificial neural networks

TL;DR

This paper introduces a real-time, low-cost statistical process monitoring method for artificial neural networks that uses data embeddings and control charts to detect nonstationarity in data streams.

Contribution

It proposes a novel monitoring approach based on embedding analysis with data depth and ranks, improving real-time detection of data drift in neural networks.

Findings

Effective detection of nonstationarity in neural network embeddings

Comparison shows improved performance over benchmark methods

Applicable across various ANN architectures and data formats

Abstract

The rapid advancement of models based on artificial intelligence demands innovative monitoring techniques which can operate in real time with low computational costs. In machine learning, especially if we consider artificial neural networks (ANNs), the models are often trained in a supervised manner. Consequently, the learned relationship between the input and the output must remain valid during the model's deployment. If this stationarity assumption holds, we can conclude that the ANN provides accurate predictions. Otherwise, the retraining or rebuilding of the model is required. We propose considering the latent feature representation of the data (called "embedding") generated by the ANN to determine the time when the data stream starts being nonstationary. In particular, we monitor embeddings by applying multivariate control charts based on the data depth calculation and normalized ranks. The performance of the introduced method is compared with benchmark approaches for various ANN architectures and different underlying data formats.