Divide-and-Conquer Meets Consensus: Unleashing the Power of Functions in Code Generation

TL;DR

FunCoder introduces a divide-and-conquer approach with functional consensus for code generation, recursively decomposing tasks into sub-functions to improve accuracy and outperform state-of-the-art models on multiple benchmarks.

Contribution

The paper presents a novel framework combining recursive function decomposition with functional consensus, enhancing code generation quality beyond existing methods.

Findings

Outperforms state-of-the-art methods by +9.8% on average across benchmarks.

Surpasses GPT-3.5 by +18.6% on smaller models like StableCode-3b.

Functional consensus is more effective than self-testing for correctness evaluation.

Abstract

Despite recent progress made by large language models in code generation, they still struggle with programs that meet complex requirements. Recent work utilizes plan-and-solve decomposition to decrease the complexity and leverage self-tests to refine the generated program. Yet, planning deep-inside requirements in advance can be challenging, and the tests need to be accurate to accomplish self-improvement. To this end, we propose FunCoder, a code generation framework incorporating the divide-and-conquer strategy with functional consensus. Specifically, FunCoder recursively branches off sub-functions as smaller goals during code generation, represented by a tree hierarchy. These sub-functions are then composited to attain more complex objectives. Additionally, we designate functions via a consensus formed by identifying similarities in program behavior, mitigating error propagation. FunCoder outperforms state-of-the-art methods by +9.8% on average in HumanEval, MBPP, xCodeEval and MATH with GPT-3.5 and GPT-4. Moreover, our method demonstrates superiority on smaller models: With FunCoder, StableCode-3b surpasses GPT-3.5 by +18.6% and achieves 97.7% of GPT-4's performance on HumanEval. Further analysis reveals that our proposed dynamic function decomposition is capable of handling complex requirements, and the functional consensus prevails over self-testing in correctness evaluation.