Fine-Tuning Small Language Models for Solution-Oriented Windows Event Log Analysis

TL;DR

This paper explores fine-tuning small language models for Windows event log analysis, demonstrating they can outperform larger models in identifying issues and suggesting solutions with less computational cost.

Contribution

It introduces a synthetic dataset and shows that fine-tuned small language models can effectively analyze logs and generate remediation actions, offering a practical, resource-efficient alternative.

Findings

Fine-tuned SLMs outperform LLMs in log analysis accuracy.

Synthetic dataset accurately reflects real-world scenarios.

SLMs require fewer resources while providing actionable insights.

Abstract

Large language models (LLMs) have shown promise for event log analysis, but their high computational requirements, reliance on cloud infrastructure, and security concerns limit practical deployment. In addition, most existing approaches focus only on the identification of the problem and do not provide actionable remediation. Small language models (SLMs) present a light-weight alternative that can be fine-tuned for a specific purpose and hosted locally. This paper investigates whether SLMs, when fine-tuned for a specific task, can serve as a practical alternative for event log analysis while also generating solutions. We first create a large-scale synthetic Windows event log dataset that contains remediation actions using a high-performing LLM. We then fine-tune multiple SLMs and LLMs using the LoRA parameter-efficient fine-tuning technique and evaluate their performance by comparing with expert assessment. The results show that the dataset accurately reflects real-world scenarios and that fine-tuned SLMs consistently outperform LLMs in identifying issues and providing relevant remediation, while requiring fewer computational resources.