Deep Learning for Source Code Modeling and Generation: Models, Applications and Challenges

TL;DR

This paper reviews deep learning techniques applied to source code modeling and generation, categorizing methods, addressing limitations, and discussing challenges and future directions in software engineering applications.

Contribution

It provides a comprehensive categorization of DL methods for source code, formulates program learning tasks within an encoder-decoder framework, and discusses recent advances and challenges.

Findings

Deep learning models have shown promising results in code understanding and generation.

Encoder-decoder frameworks effectively address source code modeling tasks.

Challenges include data quality, model interpretability, and scalability.

Abstract

Deep Learning (DL) techniques for Natural Language Processing have been evolving remarkably fast. Recently, the DL advances in language modeling, machine translation and paragraph understanding are so prominent that the potential of DL in Software Engineering cannot be overlooked, especially in the field of program learning. To facilitate further research and applications of DL in this field, we provide a comprehensive review to categorize and investigate existing DL methods for source code modeling and generation. To address the limitations of the traditional source code models, we formulate common program learning tasks under an encoder-decoder framework. After that, we introduce recent DL mechanisms suitable to solve such problems. Then, we present the state-of-the-art practices and discuss their challenges with some recommendations for practitioners and researchers as well.