RefineCoder: Iterative Improving of Large Language Models via Adaptive Critique Refinement for Code Generation

TL;DR

RefineCoder introduces an iterative self-improvement method for large language models in code generation, utilizing adaptive critique and refinement to enhance performance without relying solely on teacher model imitation.

Contribution

The paper proposes Adaptive Critique Refinement (ACR), a novel iterative self-refinement approach for LLMs that improves code quality through self-generated critique, surpassing traditional fine-tuning methods.

Findings

Achieves continuous performance improvement on multiple benchmarks.

Outperforms baselines of the same size with less data.

Demonstrates the effectiveness of self-critique in code generation.

Abstract

Code generation has attracted increasing attention with the rise of Large Language Models (LLMs). Many studies have developed powerful code LLMs by synthesizing code-related instruction data and applying supervised fine-tuning. However, these methods are limited by teacher model distillation and ignore the potential of iterative refinement by self-generated code. In this paper, we propose Adaptive Critique Refinement (ACR), which enables the model to refine itself by self-generated code and external critique, rather than directly imitating the code responses of the teacher model. Concretely, ACR includes a composite scoring system with LLM-as-a-Judge to evaluate the quality of code responses and a selective critique strategy with LLM-as-a-Critic to critique self-generated low-quality code responses. We develop the RefineCoder series by iteratively applying ACR, achieving continuous performance improvement on multiple code generation benchmarks. Compared to the baselines of the same size, our proposed RefineCoder series can achieve comparable or even superior performance using less data.