GenRL: Multimodal-foundation world models for generalization in embodied agents

TL;DR

GenRL introduces a multimodal foundation model that aligns vision-language representations with generative world models, enabling embodied agents to learn multiple tasks from prompts without extensive fine-tuning or annotations.

Contribution

This work presents a novel framework connecting foundation vision-language models with generative world models for embodied agents, facilitating task specification and learning without language annotations.

Findings

Enables multi-task generalization in locomotion and manipulation domains

Supports task specification via vision and language prompts

Introduces a data-free policy learning strategy

Abstract

Learning generalist embodied agents, able to solve multitudes of tasks in different domains is a long-standing problem. Reinforcement learning (RL) is hard to scale up as it requires a complex reward design for each task. In contrast, language can specify tasks in a more natural way. Current foundation vision-language models (VLMs) generally require fine-tuning or other adaptations to be adopted in embodied contexts, due to the significant domain gap. However, the lack of multimodal data in such domains represents an obstacle to developing foundation models for embodied applications. In this work, we overcome these problems by presenting multimodal-foundation world models, able to connect and align the representation of foundation VLMs with the latent space of generative world models for RL, without any language annotations. The resulting agent learning framework, GenRL, allows one to specify tasks through vision and/or language prompts, ground them in the embodied domain's dynamics, and learn the corresponding behaviors in imagination. As assessed through large-scale multi-task benchmarking in locomotion and manipulation domains, GenRL enables multi-task generalization from language and visual prompts. Furthermore, by introducing a data-free policy learning strategy, our approach lays the groundwork for foundational policy learning using generative world models. Website, code and data: https://mazpie.github.io/genrl/