Code Completion with Neural Attention and Pointer Networks

TL;DR

This paper introduces a neural attention and pointer mixture network to improve code completion, especially for predicting out-of-vocabulary words, by leveraging local context and a pointer copy mechanism.

Contribution

It presents a novel pointer mixture network that combines generation and copying for better OoV word prediction in code completion tasks.

Findings

The proposed model outperforms standard neural language models on benchmark datasets.

The attention mechanism enhances context understanding for code prediction.

The pointer mixture network effectively predicts OoV words in source code.

Abstract

Intelligent code completion has become an essential research task to accelerate modern software development. To facilitate effective code completion for dynamically-typed programming languages, we apply neural language models by learning from large codebases, and develop a tailored attention mechanism for code completion. However, standard neural language models even with attention mechanism cannot correctly predict the out-of-vocabulary (OoV) words that restrict the code completion performance. In this paper, inspired by the prevalence of locally repeated terms in program source code, and the recently proposed pointer copy mechanism, we propose a pointer mixture network for better predicting OoV words in code completion. Based on the context, the pointer mixture network learns to either generate a within-vocabulary word through an RNN component, or regenerate an OoV word from local context through a pointer component. Experiments on two benchmarked datasets demonstrate the effectiveness of our attention mechanism and pointer mixture network on the code completion task.