Generating Fake Cyber Threat Intelligence Using Transformer-Based Models

TL;DR

This paper demonstrates how transformer-based models can generate convincing fake cyber threat intelligence (CTI), which can be used to poison cybersecurity knowledge graphs and deceive professionals, highlighting security risks.

Contribution

It introduces a method to generate realistic fake CTI using transformers and shows its effectiveness in poisoning cyber defense systems and deceiving experts.

Findings

Fake CTI can be generated convincingly using GPT-2 with fine-tuning.

Generated fake CTI successfully poisons cybersecurity knowledge graphs.

Cybersecurity professionals often mistake fake CTI for real information.

Abstract

Cyber-defense systems are being developed to automatically ingest Cyber Threat Intelligence (CTI) that contains semi-structured data and/or text to populate knowledge graphs. A potential risk is that fake CTI can be generated and spread through Open-Source Intelligence (OSINT) communities or on the Web to effect a data poisoning attack on these systems. Adversaries can use fake CTI examples as training input to subvert cyber defense systems, forcing the model to learn incorrect inputs to serve their malicious needs. In this paper, we automatically generate fake CTI text descriptions using transformers. We show that given an initial prompt sentence, a public language model like GPT-2 with fine-tuning, can generate plausible CTI text with the ability of corrupting cyber-defense systems. We utilize the generated fake CTI text to perform a data poisoning attack on a Cybersecurity Knowledge Graph (CKG) and a cybersecurity corpus. The poisoning attack introduced adverse impacts such as returning incorrect reasoning outputs, representation poisoning, and corruption of other dependent AI-based cyber defense systems. We evaluate with traditional approaches and conduct a human evaluation study with cybersecurity professionals and threat hunters. Based on the study, professional threat hunters were equally likely to consider our fake generated CTI as true.