Prompt to Pwn: Automated Exploit Generation for Smart Contracts

TL;DR

This paper introduces ReX, an execution-grounded framework that leverages large language models to automate exploit generation for smart contracts, evaluating their effectiveness across various vulnerabilities.

Contribution

It presents ReX, a novel end-to-end system integrating LLMs with smart contract testing, and provides a comprehensive evaluation of LLM capabilities in exploit synthesis.

Findings

LLMs can generate deterministic PoCs for single-contract vulnerabilities.

Performance varies significantly by model and bug type.

Current LLMs are less effective for cross-contract attacks.

Abstract

Smart contracts are important for digital finance, yet they are hard to patch once deployed. Prior work has mainly explored LLMs for smart contract vulnerability detection, leaving end-to-end automated exploit generation (AEG) much less understood. We study that gap with \textsc{ReX}, an execution-grounded framework that links LLM-based exploit synthesis to the Foundry stack for end-to-end generation, compilation, execution, and validation. Five recent LLMs are evaluated across eight common vulnerability classes, supported by a curated dataset of 38{+} real incident PoCs and three automation aids: prompt refactoring, a compiler feedback loop, and templated test harnesses. Results indicate that current frontier LLMs can often produce deterministic PoCs for single-contract vulnerabilities, but remain weak on cross-contract attacks; outcomes depend mainly on the model and bug type, while code structure and prompt tuning contribute less in our setting. The study also surfaces important boundary conditions of LLM-driven AEG, including gaps between oracle-validated exploitability and real-world economic attacks, pointing to the need for stronger defenses and more realistic evaluation.