Hue: A User-Adaptive Parser for Hybrid Logs
Junjielong Xu, Qiuai Fu, Zhouruixing Zhu, Yutong Cheng, Zhijing Li,, Yuchi Ma, Pinjia He

TL;DR
Hue introduces a user-adaptive hybrid log parser that effectively handles both single-line and multi-line logs, outperforming existing methods and demonstrating practical deployment in real-world environments.
Contribution
Hue is the first hybrid log parser that supports both log types and incorporates user feedback for adaptive, accurate log template extraction.
Findings
Achieves 0.845 accuracy on hybrid logs, surpassing existing parsers' 0.563
Outperforms state-of-the-art on single-line log datasets
Successfully deployed in a real production environment
Abstract
Log parsing, which extracts log templates from semi-structured logs and produces structured logs, is the first and the most critical step in automated log analysis. While existing log parsers have achieved decent results, they suffer from two major limitations by design. First, they do not natively support hybrid logs that consist of both single-line logs and multi-line logs (\eg Java Exception and Hadoop Counters). Second, they fall short in integrating domain knowledge in parsing, making it hard to identify ambiguous tokens in logs. This paper defines a new research problem, \textit{hybrid log parsing}, as a superset of traditional log parsing tasks, and proposes \textit{Hue}, the first attempt for hybrid log parsing via a user-adaptive manner. Specifically, Hue converts each log message to a sequence of special wildcards using a key casting table and determines the log types via line…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware System Performance and Reliability · Data Quality and Management · Cloud Computing and Resource Management
