BAGEL: Bootstrapping Agents by Guiding Exploration with Language

TL;DR

BAGEL is a novel method that bootstraps language model agents to follow natural language instructions in digital environments by iteratively generating and refining synthetic demonstrations without human supervision, improving performance and reducing failures.

Contribution

This work introduces BAGEL, a new approach that leverages round-trip interactions between language models to generate effective training demonstrations without human input.

Findings

Achieved 2-13% absolute improvement on ToolQA and MiniWob++ benchmarks.

Reduced execution failures by up to 13 times.

Enabled effective in-context learning with synthetic demonstrations.

Abstract

Following natural language instructions by executing actions in digital environments (e.g. web-browsers and REST APIs) is a challenging task for language model (LM) agents. Unfortunately, LM agents often fail to generalize to new environments without human demonstrations. This work presents BAGEL, a method for bootstrapping LM agents without human supervision. BAGEL converts a seed set of randomly explored trajectories or synthetic instructions, into demonstrations, via round-trips between two noisy LM components: an LM labeler which converts a trajectory into a synthetic instruction, and a zero-shot LM agent which maps the synthetic instruction into a refined trajectory. By performing these round-trips iteratively, BAGEL quickly converts the initial distribution of trajectories towards those that are well-described by natural language. We use BAGEL demonstrations to adapt a zero shot LM agent at test time via in-context learning over retrieved demonstrations, and find improvements of over 2-13% absolute on ToolQA and MiniWob++, with up to 13x reduction in execution failures.