SmartLLMSentry: A Comprehensive LLM Based Smart Contract Vulnerability Detection Framework

TL;DR

SmartLLMSentry is a novel framework that uses large language models like ChatGPT to improve the detection of vulnerabilities in smart contracts, offering faster and more accurate security assessments for blockchain applications.

Contribution

It introduces a new LLM-based approach for smart contract vulnerability detection, overcoming limitations of traditional rule-based systems and demonstrating effective use of in-context training.

Findings

Achieved 91.1% exact match accuracy with sufficient data.

GPT-4 showed reduced performance compared to GPT-3 in rule generation.

Enhanced speed and accuracy in vulnerability detection.

Abstract

Smart contracts are essential for managing digital assets in blockchain networks, highlighting the need for effective security measures. This paper introduces SmartLLMSentry, a novel framework that leverages large language models (LLMs), specifically ChatGPT with in-context training, to advance smart contract vulnerability detection. Traditional rule-based frameworks have limitations in integrating new detection rules efficiently. In contrast, SmartLLMSentry utilizes LLMs to streamline this process. We created a specialized dataset of five randomly selected vulnerabilities for model training and evaluation. Our results show an exact match accuracy of 91.1% with sufficient data, although GPT-4 demonstrated reduced performance compared to GPT-3 in rule generation. This study illustrates that SmartLLMSentry significantly enhances the speed and accuracy of vulnerability detection through LLMdriven rule integration, offering a new approach to improving Blockchain security and addressing previously underexplored vulnerabilities in smart contracts.