A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source Code

TL;DR

This paper introduces a simple identifier anonymization method as a preprocessing step to effectively handle out-of-vocabulary identifiers in source code, improving deep learning model performance in code tasks.

Contribution

The paper presents a novel, easy-to-implement anonymization technique for OOV identifiers that enhances deep learning models for source code processing.

Findings

Improves Transformer performance in code completion

Enhances bug fixing accuracy

Simple preprocessing step effectively handles OOV identifiers

Abstract

There is an emerging interest in the application of natural language processing models to source code processing tasks. One of the major problems in applying deep learning to software engineering is that source code often contains a lot of rare identifiers, resulting in huge vocabularies. We propose a simple, yet effective method, based on identifier anonymization, to handle out-of-vocabulary (OOV) identifiers. Our method can be treated as a preprocessing step and, therefore, allows for easy implementation. We show that the proposed OOV anonymization method significantly improves the performance of the Transformer in two code processing tasks: code completion and bug fixing.