Tree-of-Code: A Tree-Structured Exploring Framework for End-to-End Code Generation and Execution in Complex Task Handling

TL;DR

This paper introduces Tree-of-Code, a tree-structured framework for end-to-end code generation that improves complex reasoning tasks by enabling self-growing, self-supervised code programs, significantly enhancing accuracy over previous methods like CodeAct.

Contribution

The paper proposes Tree-of-Code, a novel self-growing, self-supervised framework for code-based reasoning that addresses instability and supervision issues in prior approaches.

Findings

Tree-of-Code boosts accuracy by nearly 20% over CodeAct.

Several LLMs perform better on one-turn CodeProgram than multi-turn CodeAct.

Tree-of-Code demonstrates effective trade-offs between efficacy and efficiency.

Abstract

Solving complex reasoning tasks is a key real-world application of agents. Thanks to the pretraining of Large Language Models (LLMs) on code data, recent approaches like CodeAct successfully use code as LLM agents' action, achieving good results. However, CodeAct greedily generates the next action's code block by relying on fragmented thoughts, resulting in inconsistency and instability. Moreover, CodeAct lacks action-related ground-truth (GT), making its supervision signals and termination conditions questionable in multi-turn interactions. To address these issues, we first introduce a simple yet effective end-to-end code generation paradigm, CodeProgram, which leverages code's systematic logic to align with global reasoning and enable cohesive problem-solving. Then, we propose Tree-of-Code (ToC), which self-grows CodeProgram nodes based on the executable nature of the code and enables self-supervision in a GT-free scenario. Experimental results on two datasets using ten popular zero-shot LLMs show ToC remarkably boosts accuracy by nearly 20% over CodeAct with less than 1/4 turns. Several LLMs even perform better on one-turn CodeProgram than on multi-turn CodeAct. To further investigate the trade-off between efficacy and efficiency, we test different ToC tree sizes and exploration mechanisms. We also highlight the potential of ToC's end-to-end data generation for supervised and reinforced fine-tuning.