SCOPE: Language Models as One-Time Teacher for Hierarchical Planning in Text Environments

TL;DR

SCOPE introduces a one-shot hierarchical planning approach that pretrains a lightweight model using LLM-generated subgoals from example trajectories, significantly improving efficiency in text-based environments.

Contribution

The paper proposes SCOPE, a novel method that pretrains a hierarchical planner using LLM-derived subgoals at initialization, reducing reliance on costly LLM queries during inference.

Findings

Achieves a success rate of 0.56 on TextCraft environment.

Reduces inference time from 164.4 seconds to 3.0 seconds.

Demonstrates that suboptimal LLM-generated subgoals can still be effective.

Abstract

Long-term planning in complex, text-based environments presents significant challenges due to open-ended action spaces, ambiguous observations, and sparse feedback. Recent research suggests that large language models (LLMs) encode rich semantic knowledge about the world, which can be valuable for guiding agents in high-level reasoning and planning across both embodied and purely textual settings. However, existing approaches often depend heavily on querying LLMs during training and inference, making them computationally expensive and difficult to deploy efficiently. In addition, these methods typically employ a pretrained, unaltered LLM whose parameters remain fixed throughout training, providing no opportunity for adaptation to the target task. To address these limitations, we introduce SCOPE (Subgoal-COnditioned Pretraining for Efficient planning), a one-shot hierarchical planner that leverages LLM-generated subgoals only at initialization to pretrain a lightweight student model. Unlike prior approaches that distill LLM knowledge by repeatedly prompting the model to adaptively generate subgoals during training, our method derives subgoals directly from example trajectories. This design removes the need for repeated LLM queries, significantly improving efficiency, though at the cost of reduced explainability and potentially suboptimal subgoals. Despite their suboptimality, our results on the TextCraft environment show that LLM-generated subgoals can still serve as a strong starting point for hierarchical goal decomposition in text-based planning tasks. Compared to the LLM-based hierarchical agent ADaPT (Prasad et al., 2024), which achieves a 0.52 success rate, our method reaches 0.56 and reduces inference time from 164.4 seconds to just 3.0 seconds.