Alignment with Fill-In-the-Middle for Enhancing Code Generation

TL;DR

This paper introduces a novel fill-in-the-middle alignment method for large language models to improve code generation by splitting code into blocks and using curriculum training, validated on multiple benchmarks.

Contribution

It proposes a new approach combining AST splitting and curriculum training to enhance DPO-based code generation models, with significant experimental validation.

Findings

Improved performance on HumanEval, MBPP, APPS benchmarks

Enhanced diversity and accuracy in code generation

Effective training strategy with AST splitting and curriculum learning

Abstract

The code generation capabilities of Large Language Models (LLMs) have advanced applications like tool invocation and problem-solving. However, improving performance in code-related tasks remains challenging due to limited training data that is verifiable with accurate test cases. While Direct Preference Optimization (DPO) has shown promise, existing methods for generating test cases still face limitations. In this paper, we propose a novel approach that splits code snippets into smaller, granular blocks, creating more diverse DPO pairs from the same test cases. Additionally, we introduce the Abstract Syntax Tree (AST) splitting and curriculum training method to enhance the DPO training. Our approach demonstrates significant improvements in code generation tasks, as validated by experiments on benchmark datasets such as HumanEval (+), MBPP (+), APPS, LiveCodeBench, and BigCodeBench. Code and data are available at https://github.com/SenseLLM/StructureCoder.