Agent Lumos: Unified and Modular Training for Open-Source Language Agents

TL;DR

LUMOS is a modular, open-source framework for training language agents that improves generalization, transparency, and performance across diverse complex tasks, surpassing existing open-source and GPT-based agents.

Contribution

Introduces LUMOS, a unified, modular training framework for open-source language agents with high-quality annotations and superior performance on multiple datasets.

Findings

LUMOS outperforms larger open-source agents on unseen datasets.

LUMOS surpasses GPT agents on QA and web tasks.

LUMOS generalizes well to unseen tasks.

Abstract

Closed-source agents suffer from several issues such as a lack of affordability, transparency, and reproducibility, particularly on complex interactive tasks. This motivates the development of open-source alternatives. We introduce LUMOS, one of the first frameworks for training open-source LLM-based agents. LUMOS features a learnable, unified, and modular architecture with a planning module that learns high-level subgoal generation, and a grounding module trained to translate these into actions using various tools in the execution module. The design allows for modular upgrades and wider applicability to diverse interactive tasks. To foster generalizable agent learning, we collect large-scale, unified, and high-quality training annotations derived from diverse ground-truth reasoning rationales across various complex interactive tasks. On 9 datasets, LUMOS exhibits several key advantages: (1) LUMOS excels multiple larger open-source agents on the held-out datasets (unused for training) for each task type. LUMOS even surpasses GPT agents on QA and web tasks; (2) LUMOS outperforms open-source agents produced by chain-of-thoughts and unmodularized integrated training; and (3) LUMOS effectively generalizes to unseen tasks, outperforming 33B-scale agents and domain-specific agents.