From Context to Action: Analysis of the Impact of State Representation and Context on the Generalization of Multi-Turn Web Navigation Agents

TL;DR

This paper analyzes how context management, including interaction history and web page representation, affects the performance of LLM-based web navigation agents, especially in out-of-distribution scenarios.

Contribution

It provides a detailed analysis of contextual factors influencing web navigation agents and demonstrates improved performance through optimized context management strategies.

Findings

Enhanced agent performance on unseen websites and categories

Effective context management improves out-of-distribution generalization

Insights into designing better web navigation agents

Abstract

Recent advancements in Large Language Model (LLM)-based frameworks have extended their capabilities to complex real-world applications, such as interactive web navigation. These systems, driven by user commands, navigate web browsers to complete tasks through multi-turn dialogues, offering both innovative opportunities and significant challenges. Despite the introduction of benchmarks for conversational web navigation, a detailed understanding of the key contextual components that influence the performance of these agents remains elusive. This study aims to fill this gap by analyzing the various contextual elements crucial to the functioning of web navigation agents. We investigate the optimization of context management, focusing on the influence of interaction history and web page representation. Our work highlights improved agent performance across out-of-distribution scenarios, including unseen websites, categories, and geographic locations through effective context management. These findings provide insights into the design and optimization of LLM-based agents, enabling more accurate and effective web navigation in real-world applications.