Anomaly Detection on Seasonal Metrics via Robust Time Series Decomposition

TL;DR

This paper introduces MEDIFF, a robust real-time anomaly detection method for seasonal web service metrics using median-based decomposition and outlier testing, outperforming existing algorithms.

Contribution

The paper presents a novel robust decomposition-based anomaly detection algorithm (MEDIFF) that effectively handles seasonal and holiday effects in time series data.

Findings

MEDIFF outperforms SH-ESD and DONUT in detecting anomalies.

The method effectively handles daylight saving time and holiday effects.

Experimental results demonstrate the robustness and accuracy of MEDIFF.

Abstract

The stability and persistence of web services are important to Internet companies to improve user experience and business performances. To keep eyes on numerous metrics and report abnormal situations, time series anomaly detection methods are developed and applied by various departments in companies and institutions. In this paper, we proposed a robust anomaly detection algorithm (MEDIFF) to monitor online business metrics in real time. Specifically, a decomposition method using robust statistical metric--median--of the time series was applied to decouple the trend and seasonal components. With the effects of daylight saving time (DST) shift and holidays, corresponding components were decomposed from the time series. The residual after decomposition was tested by a generalized statistics method to detect outliers in the time series. We compared the proposed MEDIFF algorithm with two open source algorithms (SH-ESD and DONUT) by using our labeled internal business metrics. The results demonstrated the effectiveness of the proposed MEDIFF algorithm.