Labeling NIDS Rules with MITRE ATT&CK Techniques: Machine Learning vs. Large Language Models

TL;DR

This study compares Large Language Models and traditional Machine Learning in labeling NIDS rules with MITRE ATT&CK techniques, finding ML models more accurate but LLMs offer explainability and scalability for SOC use.

Contribution

It evaluates the effectiveness of LLMs versus ML models in associating NIDS rules with attack techniques, highlighting the potential of hybrid approaches.

Findings

ML models outperform LLMs in accuracy metrics

LLMs provide more explainable and scalable mappings

Hybrid approaches could enhance SOC operations

Abstract

Analysts in Security Operations Centers (SOCs) are often occupied with time-consuming investigations of alerts from Network Intrusion Detection Systems (NIDS). Many NIDS rules lack clear explanations and associations with attack techniques, complicating the alert triage and the generation of attack hypotheses. Large Language Models (LLMs) may be a promising technology to reduce the alert explainability gap by associating rules with attack techniques. In this paper, we investigate the ability of three prominent LLMs (ChatGPT, Claude, and Gemini) to reason about NIDS rules while labeling them with MITRE ATT&CK tactics and techniques. We discuss prompt design and present experiments performed with 973 Snort rules. Our results indicate that while LLMs provide explainable, scalable, and efficient initial mappings, traditional Machine Learning (ML) models consistently outperform them in accuracy, achieving higher precision, recall, and F1-scores. These results highlight the potential for hybrid LLM-ML approaches to enhance SOC operations and better address the evolving threat landscape.