LogDP: Combining Dependency and Proximity for Log-based Anomaly Detection

TL;DR

LogDP is a semi-supervised anomaly detection method that leverages dependency and proximity among log events to identify faults in large-scale systems, outperforming existing techniques.

Contribution

It introduces a novel approach combining dependency and proximity analysis for log anomaly detection, enhancing accuracy on real-world datasets.

Findings

LogDP outperforms six state-of-the-art methods in detection accuracy.

Utilizes dependency relationships and proximity to model normal log patterns.

Effective on large-scale, unlabeled log data.

Abstract

Log analysis is an important technique that engineers use for troubleshooting faults of large-scale service-oriented systems. In this study, we propose a novel semi-supervised log-based anomaly detection approach, LogDP, which utilizes the dependency relationships among log events and proximity among log sequences to detect the anomalies in massive unlabeled log data. LogDP divides log events into dependent and independent events, then learns normal patterns of dependent events using dependency and independent events using proximity. Events violating any normal pattern are identified as anomalies. By combining dependency and proximity, LogDP is able to achieve high detection accuracy. Extensive experiments have been conducted on real-world datasets, and the results show that LogDP outperforms six state-of-the-art methods.