LogLG: Weakly Supervised Log Anomaly Detection via Log-Event Graph Construction

TL;DR

LogLG introduces a weakly supervised framework for log anomaly detection that leverages log-event graphs to capture semantic keyword relationships, improving detection accuracy without extensive labeled data.

Contribution

The paper proposes a novel iterative approach using log-event graphs and self-supervised pre-training to enhance weakly supervised log anomaly detection.

Findings

Outperforms existing weakly supervised methods on five benchmarks.

Effectively captures semantic keyword relationships for anomaly detection.

Achieves significant performance improvements over prior approaches.

Abstract

Fully supervised log anomaly detection methods suffer the heavy burden of annotating massive unlabeled log data. Recently, many semi-supervised methods have been proposed to reduce annotation costs with the help of parsed templates. However, these methods consider each keyword independently, which disregards the correlation between keywords and the contextual relationships among log sequences. In this paper, we propose a novel weakly supervised log anomaly detection framework, named LogLG, to explore the semantic connections among keywords from sequences. Specifically, we design an end-to-end iterative process, where the keywords of unlabeled logs are first extracted to construct a log-event graph. Then, we build a subgraph annotator to generate pseudo labels for unlabeled log sequences. To ameliorate the annotation quality, we adopt a self-supervised task to pre-train a subgraph annotator. After that, a detection model is trained with the generated pseudo labels. Conditioned on the classification results, we re-extract the keywords from the log sequences and update the log-event graph for the next iteration. Experiments on five benchmarks validate the effectiveness of LogLG for detecting anomalies on unlabeled log data and demonstrate that LogLG, as the state-of-the-art weakly supervised method, achieves significant performance improvements compared to existing methods.