GAP-Gen: Guided Automatic Python Code Generation

TL;DR

GAP-Gen is a novel method for automatic Python code generation that uses syntactic and semantic constraints to guide fine-tuning of transformer models, resulting in improved performance over previous approaches.

Contribution

The paper introduces Syntax-Flow and Variable-Flow constraints and focuses on fine-tuning transformer models, reducing computational costs while enhancing code generation accuracy.

Findings

GAP-Gen outperforms previous methods on Python code generation benchmarks.

The use of syntactic and semantic constraints improves code quality.

Fine-tuning rather than pretraining maintains high performance with lower computational resources.

Abstract

Automatic code generation from natural language descriptions can be highly beneficial during the process of software development. In this work, we propose GAP-Gen, a Guided Automatic Python Code Generation method based on Python syntactic constraints and semantic constraints. We first introduce Python syntactic constraints in the form of Syntax-Flow, which is a simplified version of Abstract Syntax Tree (AST) reducing the size and high complexity of Abstract Syntax Tree but maintaining crucial syntactic information of Python code. In addition to Syntax-Flow, we introduce Variable-Flow which abstracts variable and function names consistently through out the code. In our work, rather than pretraining, we focus on modifying the finetuning process which reduces computational requirements but retains high generation performance on automatic Python code generation task. GAP-Gen fine-tunes the transformer based language models T5 and CodeT5 using the Code-to-Docstring datasets CodeSearchNet, CodeSearchNet AdvTest and Code-Docstring Corpus from EdinburghNLP. Our experiments show that GAP-Gen achieves better results on automatic Python code generation task than previous works.