LogRCA: Log-based Root Cause Analysis for Distributed Services

TL;DR

LogRCA is a new semi-supervised method that efficiently identifies root causes in large-scale log data, outperforming existing approaches especially for rare failures, thereby aiding faster troubleshooting in complex distributed systems.

Contribution

We introduce LogRCA, a semi-supervised learning approach for root cause analysis that handles noisy data and rare errors, with demonstrated superior performance on large-scale production logs.

Findings

Outperforms deep learning and statistical baselines in precision and recall.

Data balancing significantly improves detection of rare failures.

Effective on large-scale logs with 44.3 million entries and 80 labeled failures.

Abstract

To assist IT service developers and operators in managing their increasingly complex service landscapes, there is a growing effort to leverage artificial intelligence in operations. To speed up troubleshooting, log anomaly detection has received much attention in particular, dealing with the identification of log events that indicate the reasons for a system failure. However, faults often propagate extensively within systems, which can result in a large number of anomalies being detected by existing approaches. In this case, it can remain very challenging for users to quickly identify the actual root cause of a failure. We propose LogRCA, a novel method for identifying a minimal set of log lines that together describe a root cause. LogRCA uses a semi-supervised learning approach to deal with rare and unknown errors and is designed to handle noisy data. We evaluated our approach on a large-scale production log data set of 44.3 million log lines, which contains 80 failures, whose root causes were labeled by experts. LogRCA consistently outperforms baselines based on deep learning and statistical analysis in terms of precision and recall to detect candidate root causes. In addition, we investigated the impact of our deployed data balancing approach, demonstrating that it considerably improves performance on rare failures.