DeepAgent: A General Reasoning Agent with Scalable Toolsets

TL;DR

DeepAgent is a comprehensive reasoning agent that autonomously discovers and uses tools through an integrated process, employing a memory management system and reinforcement learning to excel across various benchmarks.

Contribution

It introduces DeepAgent, an end-to-end reasoning framework with autonomous tool discovery, a novel memory mechanism, and a reinforcement learning strategy for stable tool use.

Findings

Outperforms baselines on eight diverse benchmarks.

Effectively manages long-horizon interactions with memory folding.

Demonstrates robust general-purpose tool use in open-set scenarios.

Abstract

Large reasoning models have demonstrated strong problem-solving abilities, yet real-world tasks often require external tools and long-horizon interactions. Existing agent frameworks typically follow predefined workflows, which limit autonomous and global task completion. In this paper, we introduce DeepAgent, an end-to-end deep reasoning agent that performs autonomous thinking, tool discovery, and action execution within a single, coherent reasoning process. To manage long-horizon interactions, we introduce an autonomous memory folding mechanism that compresses past interactions into structured episodic, working, and tool memories, reducing error accumulation while preserving critical information. To teach general-purpose tool use efficiently and stably, we develop an end-to-end reinforcement learning strategy, namely ToolPO, that leverages LLM-simulated APIs and applies tool-call advantage attribution to assign fine-grained credit to the tool invocation tokens. Extensive experiments on eight benchmarks, including general tool-use tasks (ToolBench, API-Bank, TMDB, Spotify, ToolHop) and downstream applications (ALFWorld, WebShop, GAIA, HLE), demonstrate that DeepAgent consistently outperforms baselines across both labeled-tool and open-set tool retrieval scenarios. The code and demo are available at https://github.com/RUC-NLPIR/DeepAgent.