On the Embeddings of Variables in Recurrent Neural Networks for Source Code

TL;DR

This paper introduces dynamic embeddings for variables in RNNs that adapt their semantics based on context, significantly enhancing code completion and bug fixing performance.

Contribution

It proposes a novel recurrent mechanism for dynamic variable embeddings that capture contextual semantics in source code processing.

Findings

Dynamic embeddings improve RNN performance in code tasks

Significant accuracy gains in code completion and bug fixing

Context-aware variable semantics enhance model understanding

Abstract

Source code processing heavily relies on the methods widely used in natural language processing (NLP), but involves specifics that need to be taken into account to achieve higher quality. An example of this specificity is that the semantics of a variable is defined not only by its name but also by the contexts in which the variable occurs. In this work, we develop dynamic embeddings, a recurrent mechanism that adjusts the learned semantics of the variable when it obtains more information about the variable's role in the program. We show that using the proposed dynamic embeddings significantly improves the performance of the recurrent neural network, in code completion and bug fixing tasks.