AMR-Evol: Adaptive Modular Response Evolution Elicits Better Knowledge Distillation for Large Language Models in Code Generation

TL;DR

AMR-Evol introduces a two-stage adaptive response evolution framework that improves knowledge distillation quality for large language models in code generation, leading to significant performance gains on multiple benchmarks.

Contribution

This paper proposes a novel two-stage framework combining modular decomposition and adaptive response evolution to enhance response quality in knowledge distillation for code generation models.

Findings

Outperforms baseline response distillation methods on three code benchmarks.

Achieves over +3.0 points improvement on HumanEval-Plus.

Demonstrates the effectiveness of adaptive modular response evolution in LLM training.

Abstract

The impressive performance of proprietary LLMs like GPT4 in code generation has led to a trend to replicate these capabilities in open-source models through knowledge distillation (e.g. Code Evol-Instruct). However, these efforts often neglect the crucial aspect of response quality, relying heavily on teacher models for direct response distillation. This paradigm, especially for complex instructions, can degrade the quality of synthesized data, compromising the knowledge distillation process. To this end, our study introduces the Adaptive Modular Response Evolution (AMR-Evol) framework, which employs a two-stage process to refine response distillation. The first stage, modular decomposition, breaks down the direct response into more manageable sub-modules. The second stage, adaptive response evolution, automatically evolves the response with the related function modules. Our experiments with three popular code benchmarks (HumanEval, MBPP, and EvalPlus) attest to the superiority of the AMR-Evol framework over baseline response distillation methods. By comparing with the open-source Code LLMs trained on a similar scale of data, we observed performance enhancements: more than +3.0 points on HumanEval-Plus and +1.0 points on MBPP-Plus, which underscores the effectiveness of our framework. Our codes are available at https://github.com/ChiYeungLaw/AMR-Evol.