Thinking Before Running! Efficient Code Generation with Thorough Exploration and Optimal Refinement

TL;DR

ThinkCoder introduces a two-phase code generation framework combining exploration and refinement, enhanced by preference-driven optimization, achieving high accuracy with reduced computational costs on benchmarks like HumanEval and MBPP.

Contribution

The paper presents a novel framework that integrates thorough exploration, optimal refinement, and preference-driven learning to improve code generation efficiency and accuracy.

Findings

Improves Pass@1 by 3.0% over MapCoder with 6.4% of the computation cost.

Achieves higher Pass@1 than AgentCoder after fewer rounds.

Enables LLaMA2-7B to perform competitively using only 20% of resources.

Abstract

Code generation is crucial in software engineering for automating the coding process efficiently. While test-time computation methods show promise, they suffer from high latency due to multiple computation rounds. To overcome this, we introduce \textbf{ThinkCoder}, a framework that combines thorough exploration with optimal refinement. The exploration phase diversifies the solution space by searching for potential solutions, followed by a refinement phase that enhances precision. This approach allows us to select the best solution through careful consideration before taking action, avoiding excessive trial and error. To further minimize test-time computation overhead, we introduce preference-driven optimization with Reinforced Self-Training (ReST), which uses exploration trajectories from ThinkCoder to guide LLM's evolution. This approach enhances LLM's exploration efficiency via preference learning, cutting costs while maintaining accuracy. ThinkCoder boosts the performance with a single LLM, excelling on benchmarks like HumanEval and MBPP. Compared to SOTA models, it improves Pass@1 by 3.0\% over MapCoder with just 6.4\% of the computation cost. Against AgentCoder, ThinkCoder achieves a 0.5\% higher Pass@1 after 2 rounds, outperforming AgentCoder's 5 rounds. Additionally, ReST with success trajectories enhances efficiency, allowing models like LLaMA2-7B to achieve competitive results using only 20\% of the computational resources. These results highlight the framework's effectiveness and scalability.