Hierarchical Prompting Assists Large Language Model on Web Navigation

TL;DR

This paper introduces a hierarchical prompting method for large language models that improves web navigation by condensing observations, leading to a 6.2% increase in task success rate over previous methods.

Contribution

The paper proposes a novel hierarchical prompting approach that constructs action-aware summaries to enhance LLM performance in complex interactive tasks like web navigation.

Findings

Outperforms previous prompting methods by 6.2% in success rate.

Effective in tasks with long, redundant observations.

Applicable to various interactive decision-making scenarios.

Abstract

Large language models (LLMs) struggle on processing complicated observations in interactive decision making tasks. To alleviate this issue, we propose a simple hierarchical prompting approach. Diverging from previous prompting approaches that always put the full observation (e.g. a web page) to the prompt, we propose to first construct an action-aware observation which is more condensed and relevant with a dedicated SUMMARIZER prompt. The ACTOR prompt then predicts the next action based on the summarized observation. While our method has broad applicability, we particularly demonstrate its efficacy in the complex domain of web navigation where a full observation often contains redundant and irrelevant information. Our approach outperforms the previous state-of-the-art prompting mechanics by 6.2% on task success rate, demonstrating its potential on interactive decision making tasks with long observation traces.