Adaptive Thresholding Heuristic for KPI Anomaly Detection

TL;DR

This paper introduces an Adaptive Thresholding Heuristic (ATH) that dynamically adjusts anomaly detection thresholds for KPI time series, improving accuracy and reducing false positives in business contexts.

Contribution

The paper presents a novel ATH method that adapts to data changes and can be integrated with existing seasonality and outlier detection techniques.

Findings

ATH effectively reduces false positives in KPI anomaly detection.

The method is computationally efficient and scalable for real-time applications.

Experimental validation on Ericsson's KPI dataset confirms ATH's robustness.

Abstract

A plethora of outlier detectors have been explored in the time series domain, however, in a business sense, not all outliers are anomalies of interest. Existing anomaly detection solutions are confined to certain outlier detectors limiting their applicability to broader anomaly detection use cases. Network KPIs (Key Performance Indicators) tend to exhibit stochastic behaviour producing statistical outliers, most of which do not adversely affect business operations. Thus, a heuristic is required to capture the business definition of an anomaly for time series KPI. This article proposes an Adaptive Thresholding Heuristic (ATH) to dynamically adjust the detection threshold based on the local properties of the data distribution and adapt to changes in time series patterns. The heuristic derives the threshold based on the expected periodicity and the observed proportion of anomalies minimizing false positives and addressing concept drift. ATH can be used in conjunction with any underlying seasonality decomposition method and an outlier detector that yields an outlier score. This method has been tested on EON1-Cell-U, a labeled KPI anomaly dataset produced by Ericsson, to validate our hypothesis. Experimental results show that ATH is computationally efficient making it scalable for near real time anomaly detection and flexible with multiple forecasters and outlier detectors.