Length Matters: Clustering System Log Messages using Length of Words

TL;DR

This paper introduces a novel method for classifying system log messages based on word length, enabling efficient online template generation suitable for large-scale cloud systems.

Contribution

The proposed approach uses word length features for log message classification, eliminating the need for two-pass analysis and improving real-time template generation.

Findings

Supports online template generation without two-pass analysis

Efficiently handles large-scale cloud log data

Improves accuracy of log message classification

Abstract

The analysis techniques of system log messages (syslog messages) have a long history from when the syslog mechanism was invented. Typically, the analysis consists of two parts, one is a message template generation, and the other is finding something interesting using the messages classified by the inferred templates. It is important to generate better templates to achieve better, precise, or convincible analysis results. In this paper, we propose a classification methodology using the length of words of each message. Our method is suitable for online template generation because it does not require two-pass analysis to generate template messages, that is an important factor considering increasing amount of log messages produced by a large number of system components such as cloud infrastructure.