On Multi-Modal Learning of Editing Source Code

TL;DR

This paper introduces MODIT, a multi-modal neural machine translation system that incorporates developer hints, code context, and edit location to improve automatic source code editing accuracy, outperforming existing models.

Contribution

The paper presents a novel multi-modal NMT approach for code editing that leverages hints and context, significantly enhancing patch correctness and top-1 accuracy.

Findings

Hints narrow search space for patches

MODIT outperforms state-of-the-art models in top-1 accuracy

Multi-modal inputs improve code editing effectiveness

Abstract

In recent years, Neural Machine Translator (NMT) has shown promise in automatically editing source code. Typical NMT based code editor only considers the code that needs to be changed as input and suggests developers with a ranked list of patched code to choose from - where the correct one may not always be at the top of the list. While NMT based code editing systems generate a broad spectrum of plausible patches, the correct one depends on the developers' requirement and often on the context where the patch is applied. Thus, if developers provide some hints, using natural language, or providing patch context, NMT models can benefit from them. As a proof of concept, in this research, we leverage three modalities of information: edit location, edit code context, commit messages (as a proxy of developers' hint in natural language) to automatically generate edits with NMT models. To that end, we build MODIT, a multi-modal NMT based code editing engine. With in-depth investigation and analysis, we show that developers' hint as an input modality can narrow the search space for patches and outperform state-of-the-art models to generate correctly patched code in top-1 position.