An Initial Exploration of Fine-tuning Small Language Models for Smart Contract Reentrancy Vulnerability Detection

TL;DR

This paper investigates whether small language models can be effectively fine-tuned to detect reentrancy vulnerabilities in Solidity smart contracts, aiming to provide an accessible alternative to large models for security tasks.

Contribution

It presents an initial exploration into fine-tuning small language models specifically for smart contract vulnerability detection, a niche area not extensively studied before.

Findings

Small language models can be adapted for reentrancy vulnerability detection.

Fine-tuning improves the models' ability to identify bugs.

Preliminary results show promise but require further research.

Abstract

Large Language Models (LLMs) are being used more and more for various coding tasks, including to help coders identify bugs and are a promising avenue to support coders in various tasks including vulnerability detection -- particularly given the flexibility of such generative AI models and tools. Yet for many tasks it may not be suitable to use LLMs, for which it may be more suitable to use smaller language models that can fit and easily execute and train on a developer's computer. In this paper we explore and evaluate whether smaller language models can be fine-tuned to achieve reasonable results for a niche area: vulnerability detection -- specifically focusing on detecting the reentrancy bug in Solidity smart contracts.