Twin Graph-based Anomaly Detection via Attentive Multi-Modal Learning for Microservice System

TL;DR

This paper introduces MSTGAD, a semi-supervised, graph-based anomaly detection method that integrates multi-modal data using attentive neural networks to improve real-time detection in microservice systems.

Contribution

The paper presents a novel multi-modal, graph-based anomaly detection framework with transformer-based attention mechanisms for microservice systems, addressing limitations of single-modality approaches.

Findings

Effective integration of metrics, logs, and traces improves detection accuracy.

Real-time anomaly detection demonstrated on microservice system data.

Open-source implementation available for reproducibility.

Abstract

Microservice architecture has sprung up over recent years for managing enterprise applications, due to its ability to independently deploy and scale services. Despite its benefits, ensuring the reliability and safety of a microservice system remains highly challenging. Existing anomaly detection algorithms based on a single data modality (i.e., metrics, logs, or traces) fail to fully account for the complex correlations and interactions between different modalities, leading to false negatives and false alarms, whereas incorporating more data modalities can offer opportunities for further performance gain. As a fresh attempt, we propose in this paper a semi-supervised graph-based anomaly detection method, MSTGAD, which seamlessly integrates all available data modalities via attentive multi-modal learning. First, we extract and normalize features from the three modalities, and further integrate them using a graph, namely MST (microservice system twin) graph, where each node represents a service instance and the edge indicates the scheduling relationship between different service instances. The MST graph provides a virtual representation of the status and scheduling relationships among service instances of a real-world microservice system. Second, we construct a transformer-based neural network with both spatial and temporal attention mechanisms to model the inter-correlations between different modalities and temporal dependencies between the data points. This enables us to detect anomalies automatically and accurately in real-time. The source code of MSTGAD is publicly available at https://github.com/alipay/microservice_system_twin_graph_based_anomaly_detection.