AWSOM-LP: An Effective Log Parsing Technique Using Pattern Recognition and Frequency Analysis
Issam Sedki, Abdelwahab Hamou-Lhadj, Otmane Ait-Mohamed

TL;DR
AWSOM-LP is a novel log parsing tool that uses pattern recognition and frequency analysis to accurately and efficiently extract templates from large, unstructured log datasets, outperforming existing methods.
Contribution
The paper introduces AWSOM-LP, a new log parsing technique that achieves higher accuracy and efficiency compared to existing tools through innovative pattern recognition and frequency analysis methods.
Findings
Achieves 93.5% average grouping accuracy on 16 datasets.
Can generate over 80% of log templates from 10-50% of data.
Parses up to one million logs in about 5 minutes.
Abstract
Logs provide users with useful insights to help with a variety of development and operations tasks. The problem is that logs are often unstructured, making their analysis a complex task. This is mainly due to the lack of guidelines and best practices for logging, combined with a large number of logging libraries at the disposal of software developers. There exist studies that aim to parse automatically large logs. The main objective is to extract templates from samples of log data that are used to recognize future logs. In this paper, we propose AWSOM-LP, a powerful log parsing and abstraction tool, which is highly accurate, stable, and efficient. AWSOM-LP is built on the idea of applying pattern recognition and frequency analysis. First, log events are organized into patterns using a simple text processing method. Frequency analysis is then applied locally to instances of the same…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware System Performance and Reliability · Anomaly Detection Techniques and Applications · Network Security and Intrusion Detection
