XGen-Q: An Explainable Domain-Adaptive LLM Framework with Retrieval-Augmented Generation for Software Security

TL;DR

XGen-Q is an explainable, domain-adapted large language model designed for malware detection, leveraging retrieval-augmented generation and extensive training on obfuscated malware to improve generalization and interpretability in cybersecurity.

Contribution

The paper introduces XGen-Q, a novel LLM framework pretrained on malware data, employing multi-stage prompting and retrieval-augmented generation for robust, explainable malware analysis.

Findings

Achieves lower perplexity than baselines

Performs well on unseen malware samples

Provides detailed forensic reports

Abstract

Generative AI and large language models (LLMs) have shown strong capabilities in code understanding, but their use in cybersecurity, particularly for malware detection and analysis, remains limited. Existing detection systems often fail to generalize to obfuscated or previously unseen threats, underscoring the need for more adaptable and explainable models. To address this challenge, we introduce XGen-Q, a domain-adapted LLM built on the Qwen-Coder architecture and pretrained on a large-scale corpus of over one million malware samples, spanning both source and assembly code. XGen-Q uses a multi-stage prompt strategy combined with retrieval-augmented generation (RAG) to deliver reliable malware identification and detailed forensic reporting, even in the presence of complex code obfuscation. To further enhance generalization, we design a training pipeline that systematically exposes the model to diverse obfuscation patterns. Experimental results show that XGen-Q achieves significantly lower perplexity than competitive baselines and exhibits strong performance on novel malware samples, demonstrating the promise of LLM-based approaches for interpretable and robust malware analysis.