Cyber-Attack Technique Classification Using Two-Stage Trained Large Language Models

TL;DR

This paper introduces a two-stage training approach with large language models to classify cyberattack techniques from unstructured text in threat reports, improving accuracy in low-resource scenarios.

Contribution

It proposes a novel auxiliary data utilization method and a two-stage training process for better cyberattack technique classification from natural language.

Findings

Macro-F1 improved by 5-9 percentage points

Maintains competitive Micro-F1 scores

Validated on TRAM dataset with positive results

Abstract

Understanding the attack patterns associated with a cyberattack is crucial for comprehending the attacker's behaviors and implementing the right mitigation measures. However, majority of the information regarding new attacks is typically presented in unstructured text, posing significant challenges for security analysts in collecting necessary information. In this paper, we present a sentence classification system that can identify the attack techniques described in natural language sentences from cyber threat intelligence (CTI) reports. We propose a new method for utilizing auxiliary data with the same labels to improve classification for the low-resource cyberattack classification task. The system first trains the model using the augmented training data and then trains more using only the primary data. We validate our model using the TRAM data1 and the MITRE ATT&CK framework. Experiments show that our method enhances Macro-F1 by 5 to 9 percentage points and keeps Micro-F1 scores competitive when compared to the baseline performance on the TRAM dataset.