CodeRetriever: Unimodal and Bimodal Contrastive Learning for Code Search

TL;DR

CodeRetriever introduces a novel contrastive learning framework for code search, utilizing both unimodal and bimodal schemes to learn semantic representations from large-scale code and text data, achieving state-of-the-art results.

Contribution

The paper presents a new pre-training model that combines unimodal and bimodal contrastive learning for improved code semantic representation and search performance.

Findings

Achieves state-of-the-art results on eleven code search tasks.

Effective across six programming languages and various code granularities.

Significantly outperforms existing code pre-trained models.

Abstract

In this paper, we propose the CodeRetriever model, which learns the function-level code semantic representations through large-scale code-text contrastive pre-training. We adopt two contrastive learning schemes in CodeRetriever: unimodal contrastive learning and bimodal contrastive learning. For unimodal contrastive learning, we design an unsupervised learning approach to build semantic-related code pairs based on the documentation and function name. For bimodal contrastive learning, we leverage the documentation and in-line comments of code to build code-text pairs. Both contrastive objectives can fully leverage large-scale code corpus for pre-training. Extensive experimental results show that CodeRetriever achieves new state-of-the-art with significant improvement over existing code pre-trained models, on eleven domain/language-specific code search tasks with six programming languages in different code granularity (function-level, snippet-level and statement-level). These results demonstrate the effectiveness and robustness of CodeRetriever.