Semantic Sleuth: Identifying Ponzi Contracts via Large Language Models

TL;DR

This paper introduces PonziSleuth, an LLM-based method for detecting Ponzi smart contracts without requiring labeled data, achieving high accuracy and identifying new schemes in real-world blockchain data.

Contribution

PonziSleuth is the first zero-shot, LLM-driven approach for Ponzi contract detection, overcoming data limitations of previous methods.

Findings

Achieves over 96% detection accuracy with GPT-3.5-turbo.

Successfully identified 15 new Ponzi schemes from real-world contracts.

Maintains low false positive rate of 0.29% in practical deployment.

Abstract

Smart contracts, self-executing agreements directly encoded in code, are fundamental to blockchain technology, especially in decentralized finance (DeFi) and Web3. However, the rise of Ponzi schemes in smart contracts poses significant risks, leading to substantial financial losses and eroding trust in blockchain systems. Existing detection methods, such as PonziGuard, depend on large amounts of labeled data and struggle to identify unseen Ponzi schemes, limiting their reliability and generalizability. In contrast, we introduce PonziSleuth, the first LLM-driven approach for detecting Ponzi smart contracts, which requires no labeled training data. PonziSleuth utilizes advanced language understanding capabilities of LLMs to analyze smart contract source code through a novel two-step zero-shot chain-of-thought prompting technique. Our extensive evaluation on benchmark datasets and real-world contracts demonstrates that PonziSleuth delivers comparable, and often superior, performance without the extensive data requirements, achieving a balanced detection accuracy of 96.06% with GPT-3.5-turbo, 93.91% with LLAMA3, and 94.27% with Mistral. In real-world detection, PonziSleuth successfully identified 15 new Ponzi schemes from 4,597 contracts verified by Etherscan in March 2024, with a false negative rate of 0% and a false positive rate of 0.29%. These results highlight PonziSleuth's capability to detect diverse and novel Ponzi schemes, marking a significant advancement in leveraging LLMs for enhancing blockchain security and mitigating financial scams.