Language Models Meet World Models: Embodied Experiences Enhance Language Models

TL;DR

This paper introduces a method to enhance large language models by finetuning them with embodied experiences from a physical world simulator, significantly improving their reasoning and planning abilities in real-world tasks.

Contribution

The paper proposes combining world models with language models using embodied experiences, employing techniques like EWC and LoRA to retain generality and improve physical reasoning capabilities.

Findings

Improved performance on 18 downstream tasks by 64.28% on average.

Small LMs with embodied finetuning outperform larger models like ChatGPT.

Effective integration of embodied knowledge enhances reasoning and planning in LMs.

Abstract

While large language models (LMs) have shown remarkable capabilities across numerous tasks, they often struggle with simple reasoning and planning in physical environments, such as understanding object permanence or planning household activities. The limitation arises from the fact that LMs are trained only on written text and miss essential embodied knowledge and skills. In this paper, we propose a new paradigm of enhancing LMs by finetuning them with world models, to gain diverse embodied knowledge while retaining their general language capabilities. Our approach deploys an embodied agent in a world model, particularly a simulator of the physical world (VirtualHome), and acquires a diverse set of embodied experiences through both goal-oriented planning and random exploration. These experiences are then used to finetune LMs to teach diverse abilities of reasoning and acting in the physical world, e.g., planning and completing goals, object permanence and tracking, etc. Moreover, it is desirable to preserve the generality of LMs during finetuning, which facilitates generalizing the embodied knowledge across tasks rather than being tied to specific simulations. We thus further introduce the classical (EWC) for selective weight updates, combined with low-rank adapters (LoRA) for training efficiency. Extensive experiments show our approach substantially improves base LMs on 18 downstream tasks by 64.28% on average. In particular, the small LMs (1.3B, 6B, and 13B) enhanced by our approach match or even outperform much larger LMs (e.g., ChatGPT).