Automatic Smart Contract Comment Generation via Large Language Models and In-Context Learning

TL;DR

This paper introduces SCCLLM, a novel approach using large language models and in-context learning to generate smart contract comments, overcoming limitations of previous methods by leveraging domain knowledge and retrieval techniques.

Contribution

The paper proposes SCCLLM, a new method that combines LLMs and in-context learning with retrieval to improve smart contract comment generation.

Findings

SCCLLM outperforms baseline methods in automatic evaluation.

SCCLLM achieves higher quality comments in human assessments.

The approach effectively utilizes domain knowledge from a large corpus.

Abstract

The previous smart contract code comment (SCC) generation approaches can be divided into two categories: fine-tuning paradigm-based approaches and information retrieval-based approaches. However, for the fine-tuning paradigm-based approaches, the performance may be limited by the quality of the gathered dataset for the downstream task and they may have knowledge-forgetting issues. While for the information retrieval-based approaches, it is difficult for them to generate high-quality comments if similar code does not exist in the historical repository. Therefore we want to utilize the domain knowledge related to SCC generation in large language models (LLMs) to alleviate the disadvantages of these two types of approaches. In this study, we propose an approach SCCLLM based on LLMs and in-context learning. Specifically, in the demonstration selection phase, SCCLLM retrieves the top-k code snippets from the historical corpus by considering syntax, semantics, and lexical information. In the in-context learning phase, SCCLLM utilizes the retrieved code snippets as demonstrations, which can help to utilize the related knowledge for this task. We select a large corpus from a smart contract community Etherscan.io as our experimental subject. Extensive experimental results show the effectiveness of SCCLLM when compared with baselines in automatic evaluation and human evaluation.