Improving Problem Identification via Automated Log Clustering using Dimensionality Reduction

TL;DR

This paper investigates how dimensionality reduction techniques like NMF improve automated log clustering for problem identification in continuous deployment logs, demonstrating enhanced accuracy and robustness over previous methods.

Contribution

It extends prior log clustering approaches to continuous deployment logs, evaluating the impact of various dimensionality reduction and cluster merging techniques on clustering quality.

Findings

NMF significantly improves clustering accuracy and robustness.

Complete Linkage yields the best clustering performance.

Dimensionality reduction increases pipeline robustness and reduces parameter sensitivity.

Abstract

Goal: We consider the problem of automatically grouping logs of runs that failed for the same underlying reasons, so that they can be treated more effectively, and investigate the following questions: (1) Does an approach developed to identify problems in system logs generalize to identifying problems in continuous deployment logs? (2) How does dimensionality reduction affect the quality of automated log clustering? (3) How does the criterion used for merging clusters in the clustering algorithm affect clustering quality? Method: We replicate and extend earlier work on clustering system log files to assess its generalization to continuous deployment logs. We consider the optional inclusion of one of these dimensionality reduction techniques: Principal Component Analysis (PCA), Latent Semantic Indexing (LSI), and Non-negative Matrix Factorization (NMF). Moreover, we consider three alternative cluster merge criteria (Single Linkage, Average Linkage, and Weighted Linkage), in addition to the Complete Linkage criterion used in earlier work. We empirically evaluate the 16 resulting configurations on continuous deployment logs provided by our industrial collaborator. Results: Our study shows that (1) identifying problems in continuous deployment logs via clustering is feasible, (2) including NMF significantly improves overall accuracy and robustness, and (3) Complete Linkage performs best of all merge criteria analyzed. Conclusions: We conclude that problem identification via automated log clustering is improved by including dimensionality reduction, as it decreases the pipeline's sensitivity to parameter choice, thereby increasing its robustness for handling different inputs.