Addressing Leakage in Self-Supervised Contextualized Code Retrieval

TL;DR

This paper introduces a novel self-supervised contrastive learning method for contextualized code retrieval, addressing leakage issues with innovative masking and syntax alignment, and provides a new dataset for evaluation, achieving state-of-the-art results.

Contribution

It proposes a new approach to prevent leakage in self-supervised code retrieval and introduces a dataset for evaluating contextualized code retrieval.

Findings

Improved retrieval performance over existing methods

State-of-the-art results in code clone detection

Enhanced defect detection accuracy

Abstract

We address contextualized code retrieval, the search for code snippets helpful to fill gaps in a partial input program. Our approach facilitates a large-scale self-supervised contrastive training by splitting source code randomly into contexts and targets. To combat leakage between the two, we suggest a novel approach based on mutual identifier masking, dedentation, and the selection of syntax-aligned targets. Our second contribution is a new dataset for direct evaluation of contextualized code retrieval, based on a dataset of manually aligned subpassages of code clones. Our experiments demonstrate that our approach improves retrieval substantially, and yields new state-of-the-art results for code clone and defect detection.