LogTinyLLM: Tiny Large Language Models Based Contextual Log Anomaly Detection

TL;DR

This paper introduces a parameter-efficient finetuning method using LoRA for tiny large language models to improve log anomaly detection, achieving high accuracy on a large dataset with less computational cost.

Contribution

It presents a novel application of LoRA-based finetuning on tiny LLMs for contextual log anomaly detection, outperforming full finetuning methods.

Findings

LoRA finetuning improves detection accuracy by 18-19% over full finetuning.

Achieves accuracy scores between 97.76% and 98.83%.

Demonstrates effectiveness on the Thunderbird log dataset.

Abstract

Log anomaly detection using traditional rule based or deep learning based methods is often challenging due to the large volume and highly complex nature of log sequence. So effective way of detection of anomalous sequence of logs is crucial for system maintenance and development. This paper proposes parameter efficient finetuning specifically low rank adaptation (LoRA) and adapter based approaches for finding contextual anomalies in sequence of logs in large log data set. It compares different tiny large language models (LLMs) on the Thunderbird dataset. The results show that LoRA based finetuning provides substantial performance improvements of 18 to 19 percentage over LogBert based full finetuning approach, achieving accuracy scores between 97.76% and 98.83% compared to 79.37%.