LLMs for Domain Generation Algorithm Detection

TL;DR

This paper evaluates large language models for detecting domain generation algorithms, demonstrating that fine-tuning and in-context learning improve detection accuracy, especially for recent and complex DGA schemes.

Contribution

It provides a comprehensive analysis of LLM techniques for DGA detection, highlighting the effectiveness of SFT and ICL in improving detection performance.

Findings

SFT-based LLM detector achieves 94% accuracy.

ICL enables quick adaptation to new threats.

LLM methods are competitive with state-of-the-art models.

Abstract

This work analyzes the use of large language models (LLMs) for detecting domain generation algorithms (DGAs). We perform a detailed evaluation of two important techniques: In-Context Learning (ICL) and Supervised Fine-Tuning (SFT), showing how they can improve detection. SFT increases performance by using domain-specific data, whereas ICL helps the detection model to quickly adapt to new threats without requiring much retraining. We use Meta's Llama3 8B model, on a custom dataset with 68 malware families and normal domains, covering several hard-to-detect schemes, including recent word-based DGAs. Results proved that LLM-based methods can achieve competitive results in DGA detection. In particular, the SFT-based LLM DGA detector outperforms state-of-the-art models using attention layers, achieving 94% accuracy with a 4% false positive rate (FPR) and excelling at detecting word-based DGA domains.