CAM-LDS: Cyber Attack Manifestations for Automatic Interpretation of System Logs and Security Alerts

TL;DR

This paper introduces CAM-LDS, a comprehensive dataset of cyber attack logs, and demonstrates how large language models can interpret logs to identify attack techniques, addressing the limitations of manual analysis and traditional automated methods.

Contribution

The paper presents CAM-LDS, a new open-source dataset covering diverse attack scenarios, and showcases an LLM-based approach for automatic log interpretation and attack detection.

Findings

Approximately one third of attack steps are perfectly predicted by LLMs.

Another third of attack steps are predicted adequately by LLMs.

Demonstrates the potential of LLMs for semantic log analysis and attack identification.

Abstract

Log data are essential for intrusion detection and forensic investigations. However, manual log analysis is tedious due to high data volumes, heterogeneous event formats, and unstructured messages. Even though many automated methods for log analysis exist, they usually still rely on domain-specific configurations such as expert-defined detection rules, handcrafted log parsers, or manual feature-engineering. Crucially, the level of automation of conventional methods is limited due to their inability to semantically understand logs and explain their underlying causes. In contrast, Large Language Models enable domain- and format-agnostic interpretation of system logs and security alerts. Unfortunately, research on this topic remains challenging, because publicly available and labeled data sets covering a broad range of attack techniques are scarce. To address this gap, we introduce the Cyber Attack Manifestation Log Data Set (CAM-LDS), comprising seven attack scenarios that cover 81 distinct techniques across 13 tactics and collected from 18 distinct sources within a fully open-source and reproducible test environment. We extract log events that directly result from attack executions to facilitate analysis of manifestations concerning command observability, event frequencies, performance metrics, and intrusion detection alerts. We further present an illustrative case study utilizing an LLM to process the CAM-LDS. The results indicate that correct attack techniques are predicted perfectly for approximately one third of attack steps and adequately for another third, highlighting the potential of LLM-based log interpretation and utility of our data set.