Interactive Log Parsing via Light-weight User Feedback

TL;DR

This paper introduces a human-in-the-loop framework for log template mining that incorporates lightweight user feedback to improve accuracy and support interactive log analysis in web applications.

Contribution

It proposes three novel algorithms leveraging user feedback, with proven correctness and complexity bounds, enhancing existing template mining methods.

Findings

Improved template mining accuracy across multiple algorithms.

Algorithms supported by correctness and complexity guarantees.

Effective integration of user feedback in log analysis.

Abstract

Template mining is one of the foundational tasks to support log analysis, which supports the diagnosis and troubleshooting of large scale Web applications. This paper develops a human-in-the-loop template mining framework to support interactive log analysis, which is highly desirable in real-world diagnosis or troubleshooting of Web applications but yet previous template mining algorithms fails to support it. We formulate three types of light-weight user feedbacks and based on them we design three atomic human-in-the-loop template mining algorithms. We derive mild conditions under which the outputs of our proposed algorithms are provably correct. We also derive upper bounds on the computational complexity and query complexity of each algorithm. We demonstrate the versatility of our proposed algorithms by combining them to improve the template mining accuracy of five representative algorithms over sixteen widely used benchmark datasets.