Focused-DPO: Enhancing Code Generation Through Focused Preference Optimization on Error-Prone Points

TL;DR

Focuses on improving code generation accuracy by targeting error-prone points with a novel preference optimization framework, significantly reducing errors and enhancing code quality without extensive human annotations.

Contribution

Introduces Focused-DPO, a new framework that directs preference optimization to error-prone areas in code, along with an Error-Point Identification method that requires no costly annotations.

Findings

Significant improvement in code accuracy on benchmarks

Reduction in common code errors

Enhanced reliability of generated code

Abstract

Code generation models have shown significant potential for automating programming tasks. However, the challenge of generating accurate and reliable code persists due to the highly complex and long-reasoning nature of the task. Even state-of-the-art models often fail in code generation due to small errors, which can drastically affect the overall functionality of code. Our study identifies that current models tend to produce errors concentrated at specific error-prone points, which significantly impacts the accuracy of the generated code. To address this issue, we introduce Focused-DPO, a framework that enhances code generation by directing preference optimization towards these critical error-prone areas. This approach builds on Direct Preference Optimization, emphasizing accuracy in parts prone to errors. Additionally, we develop a method called Error-Point Identification, which constructs a dataset that targets these problematic points without requiring costly human annotations. Our experiments on benchmarks such as HumanEval(+), MBPP(+), and LiveCodeBench demonstrate that Focused-DPO significantly improves the precision and reliability of code generation, reducing common errors and enhancing overall code quality. By focusing on error-prone points, Focused-DPO advances the accuracy and functionality of model-generated code.