Maat: Performance Metric Anomaly Anticipation for Cloud Services with Conditional Diffusion

TL;DR

Maat introduces a two-stage approach combining metric forecasting with anomaly detection to anticipate cloud service anomalies before they occur, improving response times and accuracy.

Contribution

It is the first to address performance metric anomaly anticipation using a conditional diffusion model for multi-step forecasting in cloud services.

Findings

Outperforms real-time detectors in early anomaly prediction

Effectively forecasts abnormal metrics ahead of anomalies

Utilizes domain knowledge and incremental learning for detection

Abstract

Ensuring the reliability and user satisfaction of cloud services necessitates prompt anomaly detection followed by diagnosis. Existing techniques for anomaly detection focus solely on real-time detection, meaning that anomaly alerts are issued as soon as anomalies occur. However, anomalies can propagate and escalate into failures, making faster-than-real-time anomaly detection highly desirable for expediting downstream analysis and intervention. This paper proposes Maat, the first work to address anomaly anticipation of performance metrics in cloud services. Maat adopts a novel two-stage paradigm for anomaly anticipation, consisting of metric forecasting and anomaly detection on forecasts. The metric forecasting stage employs a conditional denoising diffusion model to enable multi-step forecasting in an auto-regressive manner. The detection stage extracts anomaly-indicating features based on domain knowledge and applies isolation forest with incremental learning to detect upcoming anomalies. Thus, our method can uncover anomalies that better conform to human expertise. Evaluation on three publicly available datasets demonstrates that Maat can anticipate anomalies faster than real-time comparatively or more effectively compared with state-of-the-art real-time anomaly detectors. We also present cases highlighting Maat's success in forecasting abnormal metrics and discovering anomalies.