GAP: Graph-Based Agent Planning with Parallel Tool Use and Reinforcement Learning

TL;DR

GAP introduces a graph-based planning framework for autonomous agents that enables adaptive parallel and serial tool execution, significantly improving efficiency and accuracy in multi-step reasoning tasks involving large language models.

Contribution

The paper presents a novel graph-based planning approach that models task dependencies to optimize parallel and sequential tool use, enhancing multi-step reasoning performance.

Findings

GAP outperforms ReAct in multi-hop question answering accuracy.

GAP achieves higher tool invocation efficiency through intelligent parallelization.

Experimental results show substantial improvements in task accuracy and efficiency.

Abstract

Autonomous agents powered by large language models (LLMs) have shown impressive capabilities in tool manipulation for complex task-solving. However, existing paradigms such as ReAct rely on sequential reasoning and execution, failing to exploit the inherent parallelism among independent sub-tasks. This sequential bottleneck leads to inefficient tool utilization and suboptimal performance in multi-step reasoning scenarios. We introduce Graph-based Agent Planning (GAP), a novel framework that explicitly models inter-task dependencies through graph-based planning to enable adaptive parallel and serial tool execution. Our approach trains agent foundation models to decompose complex tasks into dependency-aware sub-task graphs, autonomously determining which tools can be executed in parallel and which must follow sequential dependencies. This dependency-aware orchestration achieves substantial improvements in both execution efficiency and task accuracy. To train GAP, we construct a high-quality dataset of graph-based planning traces derived from the Multi-Hop Question Answering (MHQA) benchmark. We employ a two-stage training strategy: supervised fine-tuning (SFT) on the curated dataset, followed by reinforcement learning (RL) with a correctness-based reward function on strategically sampled queries where tool-based reasoning provides maximum value. Experimental results on MHQA datasets demonstrate that GAP significantly outperforms traditional ReAct baselines, particularly on multi-step retrieval tasks, while achieving dramatic improvements in tool invocation efficiency through intelligent parallelization. The project page is available at: https://github.com/WJQ7777/Graph-Agent-Planning.