Rethinking Agent Design: From Top-Down Workflows to Bottom-Up Skill Evolution

TL;DR

This paper proposes a bottom-up, experience-driven agent paradigm that enables agents to learn skills autonomously through trial, reflection, and abstraction, facilitating continual evolution in complex environments.

Contribution

It introduces a novel bottom-up agent framework that mirrors human learning, emphasizing autonomous skill acquisition and collective evolution without relying on predefined workflows.

Findings

Agents learn skills through autonomous interaction in complex games.

Skills can be rapidly shared and extended among agents.

The paradigm demonstrates effective learning in visual, open-ended environments.

Abstract

Most LLM-based agent frameworks adopt a top-down philosophy: humans decompose tasks, define workflows, and assign agents to execute each step. While effective on benchmark-style tasks, such systems rely on designer updates and overlook agents' potential to learn from experience. Recently, Silver and Sutton(2025) envision a shift into a new era, where agents could progress from a stream of experiences. In this paper, we instantiate this vision of experience-driven learning by introducing a bottom-up agent paradigm that mirrors the human learning process. Agents acquire competence through a trial-and-reasoning mechanism-exploring, reflecting on outcomes, and abstracting skills over time. Once acquired, skills can be rapidly shared and extended, enabling continual evolution rather than static replication. As more agents are deployed, their diverse experiences accelerate this collective process, making bottom-up design especially suited for open-ended environments. We evaluate this paradigm in Slay the Spire and Civilization V, where agents perceive through raw visual inputs and act via mouse outputs, the same as human players. Using a unified, game-agnostic codebase without any game-specific prompts or privileged APIs, our bottom-up agents acquire skills entirely through autonomous interaction, demonstrating the potential of the bottom-up paradigm in complex, real-world environments. Our code is available at https://github.com/AngusDujw/Bottom-Up-Agent.