A Reward-driven Automated Webshell Malicious-code Generator for Red-teaming

TL;DR

This paper introduces RAWG, a reward-driven, automated webshell malicious-code generator that uses large language models and reinforcement learning to produce diverse, obfuscated payloads for red-teaming, addressing dataset scarcity and redundancy issues.

Contribution

RAWG is a novel framework combining supervised fine-tuning and reinforcement learning to generate highly diverse and obfuscated webshell malicious code, improving over existing methods.

Findings

RAWG achieves higher payload diversity than state-of-the-art methods.

RAWG produces more effective obfuscated payloads that evade detection.

Extensive experiments validate RAWG's superior performance in red-teaming scenarios.

Abstract

Frequent cyber-attacks have elevated WebShell exploitation and defense to a critical research focus within network security. However, there remains a significant shortage of publicly available, well-categorized malicious-code datasets organized by obfuscation method. Existing malicious-code generation methods, which primarily rely on prompt engineering, often suffer from limited diversity and high redundancy in the payloads they produce. To address these limitations, we propose \textbf{RAWG}, a \textbf{R}eward-driven \textbf{A}utomated \textbf{W}ebshell Malicious-code \textbf{G}enerator designed for red-teaming applications. Our approach begins by categorizing webshell samples from common datasets into seven distinct types of obfuscation. We then employ a large language model (LLM) to extract and normalize key tokens from each sample, creating a standardized, high-quality corpus. Using this curated dataset, we perform supervised fine-tuning (SFT) on an open-source large model to enable the generation of diverse, highly obfuscated webshell malicious payloads. To further enhance generation quality, we apply Proximal Policy Optimization (PPO), treating malicious-code samples as "chosen" data and benign code as "rejected" data during reinforcement learning. Extensive experiments demonstrate that RAWG significantly outperforms current state-of-the-art methods in both payload diversity and escape effectiveness.