Accurate and fast anomaly detection in industrial processes and IoT environments

TL;DR

This paper introduces SAnD, a simple, semi-supervised anomaly detection method for industrial and IoT environments that combines statistical tools to effectively identify and interpret anomalies in complex, multicollinear signals.

Contribution

SAnD is the first procedure to integrate smoothing, variance inflation, Mahalanobis distance, thresholding, and feature importance for anomaly detection in industrial settings.

Findings

SAnD outperforms existing semi-supervised methods in accuracy.

SAnD is faster and more broadly applicable.

Effective in detecting anomalies in multicollinear, noisy signals.

Abstract

We present a novel, simple and widely applicable semi-supervised procedure for anomaly detection in industrial and IoT environments, SAnD (Simple Anomaly Detection). SAnD comprises 5 steps, each leveraging well-known statistical tools, namely; smoothing filters, variance inflation factors, the Mahalanobis distance, threshold selection algorithms and feature importance techniques. To our knowledge, SAnD is the first procedure that integrates these tools to identify anomalies and help decipher their putative causes. We show how each step contributes to tackling technical challenges that practitioners face when detecting anomalies in industrial contexts, where signals can be highly multicollinear, have unknown distributions, and intertwine short-lived noise with the long(er)-lived actual anomalies. The development of SAnD was motivated by a concrete case study from our industrial partner, which we use here to show its effectiveness. We also evaluate the performance of SAnD by comparing it with a selection of semi-supervised methods on public datasets from the literature on anomaly detection. We conclude that SAnD is effective, broadly applicable, and outperforms existing approaches in both anomaly detection and runtime.