CoDesc: A Large Code-Description Parallel Dataset

TL;DR

CoDesc is a large, noise-cleaned dataset of 4.2 million Java methods and descriptions that advances code summarization, search, and pretraining for Java language models.

Contribution

This paper introduces CoDesc, the largest parallel code-description dataset, with noise removal and benchmarks, enabling significant improvements in code understanding tasks.

Findings

Improves code search accuracy by up to 22%

Sets new state-of-the-art in code summarization

Enhances pretraining and fine-tuning for Java models

Abstract

Translation between natural language and source code can help software development by enabling developers to comprehend, ideate, search, and write computer programs in natural language. Despite growing interest from the industry and the research community, this task is often difficult due to the lack of large standard datasets suitable for training deep neural models, standard noise removal methods, and evaluation benchmarks. This leaves researchers to collect new small-scale datasets, resulting in inconsistencies across published works. In this study, we present CoDesc -- a large parallel dataset composed of 4.2 million Java methods and natural language descriptions. With extensive analysis, we identify and remove prevailing noise patterns from the dataset. We demonstrate the proficiency of CoDesc in two complementary tasks for code-description pairs: code summarization and code search. We show that the dataset helps improve code search by up to 22\% and achieves the new state-of-the-art in code summarization. Furthermore, we show CoDesc's effectiveness in pre-training--fine-tuning setup, opening possibilities in building pretrained language models for Java. To facilitate future research, we release the dataset, a data processing tool, and a benchmark at \url{https://github.com/csebuetnlp/CoDesc}.