RedShell: A Generative AI-Based Approach to Ethical Hacking

TL;DR

RedShell is a novel tool that leverages generative AI to ethically produce malicious PowerShell code for cybersecurity testing, demonstrating high syntactic validity and semantic consistency.

Contribution

This work introduces RedShell, a specialized generative model for malicious PowerShell code, along with a new dataset for training and evaluating offensive code generation models.

Findings

RedShell generates syntactically valid PowerShell with less than 10% parse errors.

Generated samples achieve over 50% similarity on Edit Distance and 40% on METEOR.

The approach advances AI-assisted pentesting within controlled, ethical environments.

Abstract

The application of Machine Learning techniques in code generation is now a common practice for most developers. Tools such as ChatGPT from OpenAI leverage the natural language processing capabilities of Large Language Models to generate machine code from natural language descriptions. In the cybersecurity field, red teams can also take advantage of generative models to build malicious code generators, providing more automation to Pentest audits. However, the application of Large Language Models in malicious code generation remains challenging due to the lack of data to train and evaluate offensive code generators. In this work, we propose RedShell, a tool that allows ethical hackers to generate malicious PowerShell code. We also introduce a ground truth dataset, combining publicly available code samples to fine-tune models in malicious PowerShell generation. Our experiments demonstrate the strong capabilities of RedShell in generating syntactically valid PowerShell, with fewer than 10% of the generated samples resulting in parse errors. Furthermore, our specialized model was able to produce samples that were semantically consistent with reference snippets, achieving a competitive performance on standard output similarity metrics such as Edit Distance and METEOR, with their mean similarity scores exceeding 50% and 40%, respectively. This work sheds light on the state-of-the-art research in the field of Generative AI applied to Pentesting, and also serves as a steppingstone for future advancements, highlighting the potential benefits these models hold within such controlled environments.