Evaluation of Contrastive Learning with Various Code Representations for Code Clone Detection

TL;DR

This paper evaluates contrastive learning algorithms combined with different code representations for code clone and plagiarism detection, finding graph-based models and specific CL algorithms like SimCLR and SwAV most effective.

Contribution

It systematically compares popular contrastive learning algorithms with various code representations on clone and plagiarism detection tasks, introducing a new dataset for plagiarism detection.

Findings

Graph-based models outperform others in both tasks.

SimCLR and SwAV yield better results among CL algorithms.

Moco demonstrates the most robustness across tasks.

Abstract

Code clones are pairs of code snippets that implement similar functionality. Clone detection is a fundamental branch of automatic source code comprehension, having many applications in refactoring recommendation, plagiarism detection, and code summarization. A particularly interesting case of clone detection is the detection of semantic clones, i.e., code snippets that have the same functionality but significantly differ in implementation. A promising approach to detecting semantic clones is contrastive learning (CL), a machine learning paradigm popular in computer vision but not yet commonly adopted for code processing. Our work aims to evaluate the most popular CL algorithms combined with three source code representations on two tasks. The first task is code clone detection, which we evaluate on the POJ-104 dataset containing implementations of 104 algorithms. The second task is plagiarism detection. To evaluate the models on this task, we introduce CodeTransformator, a tool for transforming source code. We use it to create a dataset that mimics plagiarised code based on competitive programming solutions. We trained nine models for both tasks and compared them with six existing approaches, including traditional tools and modern pre-trained neural models. The results of our evaluation show that proposed models perform diversely in each task, however the performance of the graph-based models is generally above the others. Among CL algorithms, SimCLR and SwAV lead to better results, while Moco is the most robust approach. Our code and trained models are available at https://doi.org/10.5281/zenodo.6360627, https://doi.org/10.5281/zenodo.5596345.