Boost, Disentangle, and Customize: A Robust System2-to-System1 Pipeline for Code Generation

TL;DR

This paper introduces a robust System2-to-System1 pipeline for code generation using a novel BDC framework, combining boosting, data disentanglement, and customization to improve reasoning and adaptability of LLMs.

Contribution

The paper proposes a new BDC framework with MC-Tree-Of-Agents, DisenLora algorithm, and input-aware hypernetworks to enhance LLM reasoning, robustness, and customization in code generation tasks.

Findings

Effective exploration of System 2 knowledge via mutual boosting.

Disentangling heterogeneous data improves LLM training.

Customized problem solvers enhance code generation accuracy.

Abstract

Large language models (LLMs) have demonstrated remarkable capabilities in various domains, particularly in system 1 tasks, yet the intricacies of their problem-solving mechanisms in system 2 tasks are not sufficiently explored. Recent research on System2-to-System1 methods surge, exploring the System 2 reasoning knowledge via inference-time computation and compressing the explored knowledge into System 1 process. In this paper, we focus on code generation, which is a representative System 2 task, and identify two primary challenges: (1) the complex hidden reasoning processes and (2) the heterogeneous data distributions that complicate the exploration and training of robust LLM solvers. To tackle these issues, we propose a novel BDC framework that explores insightful System 2 knowledge of LLMs using a MC-Tree-Of-Agents algorithm with mutual \textbf{B}oosting, \textbf{D}isentangles the heterogeneous training data for composable LoRA-experts, and obtain \textbf{C}ustomized problem solver for each data instance with an input-aware hypernetwork to weight over the LoRA-experts, offering effectiveness, flexibility, and robustness. This framework leverages multiple LLMs through mutual verification and boosting, integrated into a Monte-Carlo Tree Search process enhanced by reflection-based pruning and refinement. Additionally, we introduce the DisenLora algorithm, which clusters heterogeneous data to fine-tune LLMs into composable Lora experts, enabling the adaptive generation of customized problem solvers through an input-aware hypernetwork. This work lays the groundwork for advancing LLM capabilities in complex reasoning tasks, offering a novel System2-to-System1 solution.