Evaluation of Siamese Networks for Semantic Code Search

TL;DR

This paper evaluates Siamese networks for semantic code search, demonstrating their effectiveness in learning joint representations of code and natural language, leading to improved search performance across multiple datasets.

Contribution

It explores multiple architectures of Siamese networks for code search, showing their potential to enhance semantic understanding and outperform previous baselines.

Findings

Siamese networks act as strong regularizers for code and text embedding extraction.

They achieve better performance than previous methods on two programming languages.

Analysis of embedding spaces provides insights for future improvements.

Abstract

With the increase in the number of open repositories and discussion forums, the use of natural language for semantic code search has become increasingly common. The accuracy of the results returned by such systems, however, can be low due to 1) limited shared vocabulary between code and user query and 2) inadequate semantic understanding of user query and its relation to code syntax. Siamese networks are well suited to learning such joint relations between data, but have not been explored in the context of code search. In this work, we evaluate Siamese networks for this task by exploring multiple extraction network architectures. These networks independently process code and text descriptions before passing them to a Siamese network to learn embeddings in a common space. We experiment on two different datasets and discover that Siamese networks can act as strong regularizers on networks that extract rich information from code and text, which in turn helps achieve impressive performance on code search beating previous baselines on $2$ programming languages. We also analyze the embedding space of these networks and provide directions to fully leverage the power of Siamese networks for semantic code search.