Building an Automated and Self-Aware Anomaly Detection System

TL;DR

This paper presents an automated, self-aware anomaly detection system for time series data that adapts its models dynamically, reducing manual effort and improving detection accuracy across diverse scenarios.

Contribution

The paper introduces a novel anomaly detection system that autonomously monitors and adjusts its models, enhancing robustness and reducing the need for manual tuning.

Findings

Outperforms existing methods on benchmark datasets

Adapts to changing data patterns automatically

Reduces false positives and manual intervention

Abstract

Organizations rely heavily on time series metrics to measure and model key aspects of operational and business performance. The ability to reliably detect issues with these metrics is imperative to identifying early indicators of major problems before they become pervasive. It can be very challenging to proactively monitor a large number of diverse and constantly changing time series for anomalies, so there are often gaps in monitoring coverage, disabled or ignored monitors due to false positive alarms, and teams resorting to manual inspection of charts to catch problems. Traditionally, variations in the data generation processes and patterns have required strong modeling expertise to create models that accurately flag anomalies. In this paper, we describe an anomaly detection system that overcomes this common challenge by keeping track of its own performance and making changes as necessary to each model without requiring manual intervention. We demonstrate that this novel approach outperforms available alternatives on benchmark datasets in many scenarios.