Log Parsing using LLMs with Self-Generated In-Context Learning and Self-Correction

TL;DR

AdaParser leverages large language models with self-generated in-context learning and self-correction to improve log parsing accuracy, especially on evolving data, outperforming existing methods in various scenarios.

Contribution

The paper introduces AdaParser, a novel framework that uses LLMs with self-generated in-context learning and self-correction, including a template corrector and dynamic candidate set, to enhance log parsing accuracy.

Findings

Outperforms state-of-the-art log parsers on large-scale datasets.

Effectively handles evolving log data with dynamic template adaptation.

Significantly improves LLM performance in zero-shot log parsing scenarios.

Abstract

Log parsing transforms log messages into structured formats, serving as a crucial step for log analysis. Despite a variety of log parsers that have been proposed, their performance on evolving log data remains unsatisfactory due to reliance on human-crafted rules or learning-based models with limited training data. The recent emergence of large language models (LLMs) has demonstrated strong abilities in understanding natural language and code, making it promising to apply LLMs for log parsing. Consequently, several studies have proposed LLM-based log parsers. However, LLMs may produce inaccurate templates, and existing LLM-based log parsers directly use the template generated by the LLM as the parsing result, hindering the accuracy of log parsing. Furthermore, these log parsers depend heavily on historical log data as demonstrations, which poses challenges in maintaining accuracy when dealing with scarce historical log data or evolving log data. To address these challenges, we propose AdaParser, an effective and adaptive log parsing framework using LLMs with self-generated in-context learning (SG-ICL) and self-correction. To facilitate accurate log parsing, AdaParser incorporates a novel component, a template corrector, which utilizes the LLM to correct potential parsing errors in the templates it generates. In addition, AdaParser maintains a dynamic candidate set composed of previously generated templates as demonstrations to adapt evolving log data. Extensive experiments on public large-scale datasets indicate that AdaParser outperforms state-of-the-art methods across all metrics, even in zero-shot scenarios. Moreover, when integrated with different LLMs, AdaParser consistently enhances the performance of the utilized LLMs by a large margin.