"The tail wags the dog": A study of anomaly detection in commercial application performance

TL;DR

This paper presents a platform-agnostic, black box queuing model-based approach for detecting application performance anomalies across diverse workloads and architectures, enhancing systems management in the IT industry.

Contribution

It introduces a novel performance signature modeling technique using M/M/1 queues and curve fitting, applicable to various application types without customization.

Findings

Accurately models system performance signatures across workloads

Detects slow down events effectively in diverse environments

Platform and architecture agnostic detection method

Abstract

The IT industry needs systems management models that leverage available application information to detect quality of service, scalability and health of service. Ideally this technique would be common for varying application types with different n-tier architectures under normal production conditions of varying load, user session traffic, transaction type, transaction mix, and hosting environment. This paper shows that a whole of service measurement paradigm utilizing a black box M/M/1 queuing model and auto regression curve fitting of the associated CDF are an accurate model to characterize system performance signatures. This modeling method is also used to detect application slow down events. The technique was shown to work for a diverse range of workloads ranging from 76 Tx/ 5min to 19,025 Tx/ 5min. The method did not rely on customizations specific to the n-tier architecture of the systems being analyzed and so the performance anomaly detection technique was shown to be platform and configuration agnostic.