Instruction-Tuned LLMs for Parsing and Mining Unstructured Logs on Leadership HPC Systems

TL;DR

This paper introduces a domain-adapted, instruction-following LLM framework for parsing and mining unstructured HPC system logs, achieving high accuracy and practical utility in large-scale log analysis.

Contribution

It presents a hybrid fine-tuning methodology for adapting LLMs to HPC log data, enabling efficient, privacy-preserving log parsing comparable to larger models.

Findings

Achieves parsing accuracy comparable to larger models like LLaMA 70B.

Successfully analyzed over 600 million production logs from Frontier supercomputer.

Uncovered operational patterns, anomalies, and workload-error correlations.

Abstract

Leadership-class HPC systems generate massive volumes of heterogeneous, largely unstructured system logs. Because these logs originate from diverse software, hardware, and runtime layers, they exhibit inconsistent formats, making structure extraction and pattern discovery extremely challenging. Therefore, robust log parsing and mining is critical to transform this raw telemetry into actionable insights that reveal operational patterns, diagnose anomalies, and enable reliable, efficient, and scalable system analysis. Recent advances in large language models (LLMs) offer a promising new direction for automated log understanding in leadership-class HPC environments. To capitalize on this opportunity, we present a domain-adapted, instruction-following, LLM-driven framework that leverages chain-of-thought (CoT) reasoning to parse and structure HPC logs with high fidelity. Our approach combines domain-specific log-template data with instruction-tuned examples to fine-tune an 8B-parameter LLaMA model tailored for HPC log analysis. We develop a hybrid fine-tuning methodology that adapts a general-purpose LLM to domain-specific log data, enabling privacy-preserving, locally deployable, fast, and energy-efficient log-mining approach. We conduct experiments on a diverse set of log datasets from the LogHub repository. The evaluation confirms that our approach achieves parsing accuracy on par with significantly larger models, such as LLaMA 70B and Anthropic's Claude. We further validate the practical utility of our fine-tuned LLM model by parsing over 600 million production logs from the Frontier supercomputer over a four-week window, uncovering critical patterns in temporal dynamics, node-level anomalies, and workload-error log correlations.