Prompting for Automatic Log Template Extraction
Junjielong Xu, Ruichun Yang, Yintong Huo, Chengyu Zhang, and Pinjia He

TL;DR
DivLog leverages large language models and in-context learning to extract log templates effectively without training, outperforming traditional parsers on diverse datasets.
Contribution
The paper introduces DivLog, a novel log parsing framework using LLMs and in-context learning, eliminating the need for model tuning or handcrafted features.
Findings
Achieves 98.1% parsing accuracy on public datasets
Attains over 92% in template precision and recall
Outperforms existing log parsers with state-of-the-art results
Abstract
Log parsing, which involves log template extraction from semi-structured logs to produce structured logs, is the first and the most critical step in automated log analysis. However, current log parsers suffer from limited effectiveness for two reasons. First, traditional data-driven log parsers solely rely on heuristics or handcrafted features designed by domain experts, which may not consistently perform well on logs from diverse systems. Second, existing supervised log parsers require model tuning, which is often limited to fixed training samples and causes sub-optimal performance across the entire log source. To address this limitation, we propose DivLog, an effective log parsing framework based on the in-context learning (ICL) ability of large language models (LLMs). Specifically, before log parsing, DivLog samples a small amount of offline logs as candidates by maximizing their…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware System Performance and Reliability · Data Quality and Management · Traffic Prediction and Management Techniques
