Evaluating Language Models For Threat Detection in IoT Security Logs

TL;DR

This paper explores the use of fine-tuned large language models for detecting threats and recommending mitigations in IoT security logs, showing improved multi-class attack classification over classical methods.

Contribution

It introduces a pipeline leveraging LLMs for anomaly detection and mitigation in IoT logs, comparing different strategies and mapping threats to mitigation actions.

Findings

LLMs outperform classical models in multi-class attack detection

Fine-tuning improves threat detection and mitigation recommendations

Mapping threats to MITRE CAPEC enhances actionable insights

Abstract

Log analysis is a relevant research field in cybersecurity as they can provide a source of information for the detection of threats to networks and systems. This paper presents a pipeline to use fine-tuned Large Language Models (LLMs) for anomaly detection and mitigation recommendation using IoT security logs. Utilizing classical machine learning classifiers as a baseline, three open-source LLMs are compared for binary and multiclass anomaly detection, with three strategies: zero-shot, few-shot prompting and fine-tuning using an IoT dataset. LLMs give better results on multi-class attack classification than the corresponding baseline models. By mapping detected threats to MITRE CAPEC, defining a set of IoT-specific mitigation actions, and fine-tuning the models with those actions, the models are able to provide a combined detection and recommendation guidance.