UniGenCoder: Merging Seq2Seq and Seq2Tree Paradigms for Unified Code Generation

TL;DR

UniGenCoder unifies sequence-to-sequence and sequence-to-tree paradigms for code generation, improving performance by dynamically selecting the best approach for each task through multi-task learning, distillation, and contrastive training.

Contribution

This paper introduces a novel unified model that combines two complementary code generation paradigms with a dynamic selector, enhancing flexibility and accuracy.

Findings

Effective on text-to-code and code-to-code tasks

Outperforms existing models in accuracy

Demonstrates the benefit of paradigm integration

Abstract

Deep learning-based code generation has completely transformed the way developers write programs today. Existing approaches to code generation have focused either on the Sequence-to-Sequence paradigm, which generates target code as a sequence of tokens, or the Sequence-to-Tree paradigm, which outputs code as a sequence of actions. While these two paradigms are intuitively complementary, their combination has not been previously explored. By comparing the code generated under these two paradigms, we find that integrating them holds significant potential. In this paper, we propose UniGenCoder for code-related generation tasks, which consists of a shared encoder, a shared decoder with a minimal set of additional parameters to unify two paradigms, and a selector that dynamically chooses optimal paradigm for each instance. Also, during the model training, we first perform the multi-task learning and distillation strategies to facilitate knowledge transfer between two paradigms, and then leverage contrastive learning to train the selector. Experimental results on the text-to-code and code-to-code generation tasks demonstrate the effectiveness of our proposed model. We release our code at https://github.com/DeepLearnXMU/UniGenCoder.