Efficacy of Various Large Language Models in Generating Smart Contracts

TL;DR

This paper evaluates how well large language models generate secure and efficient smart contracts in Solidity, revealing their strengths and limitations, and introduces new prompting strategies for improved contract generation.

Contribution

It provides an extensive analysis of LLMs' ability to generate smart contracts with security and efficiency, and proposes novel prompting techniques for better results.

Findings

LLMs often struggle with security details in smart contracts

Many LLMs successfully generate common smart contract types

New prompting strategies improve contract generation quality

Abstract

This study analyzes the application of code-generating Large Language Models in the creation of immutable Solidity smart contracts on the Ethereum Blockchain. Other works have previously analyzed Artificial Intelligence code generation abilities. This paper aims to expand this to a larger scope to include programs where security and efficiency are of utmost priority such as smart contracts. The hypothesis leading into the study was that LLMs in general would have difficulty in rigorously implementing security details in the code, which was shown through our results, but surprisingly generally succeeded in many common types of contracts. We also discovered a novel way of generating smart contracts through new prompting strategies.