Actionable Cyber Threat Intelligence using Knowledge Graphs and Large Language Models

TL;DR

This paper presents a novel approach combining Large Language Models and Knowledge Graphs to automate the extraction and structuring of actionable cyber threat intelligence from unstructured data, enhancing cybersecurity decision-making.

Contribution

It introduces a methodology using open-source LLMs and fine-tuning techniques to improve CTI extraction and KG construction, addressing scalability challenges.

Findings

Prompt engineering and fine-tuning outperform basic prompt methods

The approach effectively extracts relevant threat information

Scaling to large datasets remains a challenge

Abstract

Cyber threats are constantly evolving. Extracting actionable insights from unstructured Cyber Threat Intelligence (CTI) data is essential to guide cybersecurity decisions. Increasingly, organizations like Microsoft, Trend Micro, and CrowdStrike are using generative AI to facilitate CTI extraction. This paper addresses the challenge of automating the extraction of actionable CTI using advancements in Large Language Models (LLMs) and Knowledge Graphs (KGs). We explore the application of state-of-the-art open-source LLMs, including the Llama 2 series, Mistral 7B Instruct, and Zephyr for extracting meaningful triples from CTI texts. Our methodology evaluates techniques such as prompt engineering, the guidance framework, and fine-tuning to optimize information extraction and structuring. The extracted data is then utilized to construct a KG, offering a structured and queryable representation of threat intelligence. Experimental results demonstrate the effectiveness of our approach in extracting relevant information, with guidance and fine-tuning showing superior performance over prompt engineering. However, while our methods prove effective in small-scale tests, applying LLMs to large-scale data for KG construction and Link Prediction presents ongoing challenges.