AutoGuide: Automated Generation and Selection of Context-Aware Guidelines for Large Language Model Agents

TL;DR

AutoGuide introduces a framework that automatically generates concise, context-aware guidelines from offline experiences to improve large language model agents' performance in unfamiliar domains like web navigation.

Contribution

It presents a novel method for automatically creating natural language, context-specific guidelines to enhance LLM agent decision-making beyond demonstration-based learning.

Findings

AutoGuide outperforms baselines in web navigation tasks.

Guidelines improve decision relevance and accuracy.

Framework effectively generalizes to complex domains.

Abstract

Recent advances in large language models (LLMs) have empowered AI agents capable of performing various sequential decision-making tasks. However, effectively guiding LLMs to perform well in unfamiliar domains like web navigation, where they lack sufficient knowledge, has proven to be difficult with the demonstration-based in-context learning paradigm. In this paper, we introduce a novel framework, called AutoGuide, which addresses this limitation by automatically generating context-aware guidelines from offline experiences. Importantly, each context-aware guideline is expressed in concise natural language and follows a conditional structure, clearly describing the context where it is applicable. As a result, our guidelines facilitate the provision of relevant knowledge for the agent's current decision-making process, overcoming the limitations of the conventional demonstration-based learning paradigm. Our evaluation demonstrates that AutoGuide significantly outperforms competitive baselines in complex benchmark domains, including real-world web navigation.