Enhancing Web Service Anomaly Detection via Fine-grained Multi-modal Association and Frequency Domain Analysis

TL;DR

This paper introduces FFAD, a novel anomaly detection method for web services that uses graph-based alignment and Fourier frequency analysis to improve detection accuracy over existing methods.

Contribution

The paper proposes a new approach combining graph-based association and frequency domain analysis to enhance web service anomaly detection accuracy.

Findings

Achieves an average F1-score of 93.6% on real-world datasets.

Outperforms previous state-of-the-art methods by 8.8%.

Effectively addresses issues of asynchrony and overgeneralization in anomaly detection.

Abstract

Anomaly detection is crucial for ensuring the stability and reliability of web service systems. Logs and metrics contain multiple information that can reflect the system's operational state and potential anomalies. Thus, existing anomaly detection methods use logs and metrics to detect web service systems' anomalies through data fusion approaches. They associate logs and metrics using coarse-grained time window alignment and capture the normal patterns of system operation through reconstruction. However, these methods have two issues that limit their performance in anomaly detection. First, due to asynchrony between logs and metrics, coarse-grained time window alignment cannot achieve a precise association between the two modalities. Second, reconstruction-based methods suffer from severe overgeneralization problems, resulting in anomalies being accurately reconstructed. In this paper, we propose a novel anomaly detection method named FFAD to address these two issues. On the one hand, FFAD employs graph-based alignment to mine and extract associations between the modalities from the constructed log-metric relation graph, achieving precise associations between logs and metrics. On the other hand, we improve the model's fit to normal data distributions through Fourier Frequency Focus, thereby enhancing the effectiveness of anomaly detection. We validated the effectiveness of our model on two real-world industrial datasets and one open-source dataset. The results show that our method achieves an average anomaly detection F1-score of 93.6%, representing an 8.8% improvement over previous state-of-the-art methods.