One STEP at a time: Language Agents are Stepwise Planners

TL;DR

This paper introduces STEP, a framework that improves language agents' planning abilities by learning from past experiences, leading to better task performance in dynamic environments.

Contribution

The paper presents a novel framework, STEP, that enhances language agents' planning by integrating experience-based learning and structured components for task decomposition, action generation, evaluation, and memory.

Findings

STEP outperforms state-of-the-art models on ScienceWorld benchmark

Achieves an overall score of 67.4 and completes 12 out of 18 tasks

Demonstrates improved planning and task-solving in dynamic environments

Abstract

Language agents have shown promising adaptability in dynamic environments to perform complex tasks. However, despite the versatile knowledge embedded in large language models, these agents still fall short when it comes to tasks that require planning. We introduce STEP, a novel framework designed to efficiently learn from previous experiences to enhance the planning capabilities of language agents in future steps. Concretely, STEP functions through four interconnected components. First, the Planner takes on the task, breaks it down into subtasks and provides relevant insights. Then the Executor generates action candidates, while the Evaluator ensures the actions align with learned rules from previous experiences. Lastly, Memory stores experiences to inform future decisions. In the ScienceWorld benchmark, our results show that STEP consistently outperforms state-of-the-art models, achieving an overall score of 67.4 and successfully completing 12 out of 18 tasks. These findings highlight STEP's potential as a framework for enhancing planning capabilities in language agents, paving the way for more sophisticated task-solving in dynamic environments.