TPLogAD: Unsupervised Log Anomaly Detection Based on Event Templates and Key Parameters

TL;DR

TPLogAD is an unsupervised log anomaly detection method that uses semantic representations of event templates and key parameters to improve accuracy and robustness over existing techniques.

Contribution

It introduces itemplate2vec and para2vec, novel semantic embedding methods for logs, enabling effective anomaly detection without supervision.

Findings

Outperforms existing log anomaly detection methods on public datasets

Effectively handles log diversity and dynamics

Provides a universal approach for unstructured log analysis

Abstract

Log-system is an important mechanism for recording the runtime status and events of Web service systems, and anomaly detection in logs is an effective method of detecting problems. However, manual anomaly detection in logs is inefficient, error-prone, and unrealistic. Existing log anomaly detection methods either use the indexes of event templates, or form vectors by embedding the fixed string part of the template as a sentence, or use time parameters for sequence analysis. However, log entries often contain features and semantic information that cannot be fully represented by these methods, resulting in missed and false alarms. In this paper, we propose TPLogAD, a universal unsupervised method for analyzing unstructured logs, which performs anomaly detection based on event templates and key parameters. The itemplate2vec and para2vec included in TPLogAD are two efficient and easy-to-implement semantic representation methods for logs, detecting anomalies in event templates and parameters respectively, which has not been achieved in previous work. Additionally, TPLogAD can avoid the interference of log diversity and dynamics on anomaly detection. Our experiments on four public log datasets show that TPLogAD outperforms existing log anomaly detection methods.