LEMON: Language-Based Environment Manipulation via Execution-Guided Pre-training

TL;DR

LEMON introduces a versatile, pre-trained language model framework for environment manipulation tasks that generalizes across diverse environments and achieves state-of-the-art results through execution-guided pre-training.

Contribution

The paper presents a task-agnostic approach and an execution-guided pre-training strategy that significantly improve environment manipulation performance across multiple tasks.

Findings

Achieves new state-of-the-art on four tasks

Execution-guided pre-training improves performance on all tasks

Demonstrates generalization across diverse environments

Abstract

Language-based environment manipulation requires agents to manipulate the environment following natural language instructions, which is challenging due to the huge space of the environments. To address this challenge, various approaches have been proposed in recent work. Although these approaches work well for their intended environments, they are difficult to generalize across environments. In this work, we propose LEMON, a general framework for language-based environment manipulation tasks. Specifically, we first specify a task-agnostic approach for language-based environment manipulation tasks, which can deal with various environments using the same generative language model. Then we propose an execution-guided pre-training strategy to inject prior knowledge of environments to the language model with a pure synthetic pre-training corpus. Experimental results on tasks including Alchemy, Scene, Tangrams, ProPara and Recipes demonstrate the effectiveness of LEMON: it achieves new state-of-the-art results on four of the tasks, and the execution-guided pre-training strategy brings remarkable improvements on all experimental tasks.