LIMT: Language-Informed Multi-Task Visual World Models

TL;DR

This paper introduces a model-based multi-task reinforcement learning approach that uses pre-trained language models to create semantic task representations, improving multi-task learning efficiency and effectiveness in robotic applications.

Contribution

It proposes a novel method leveraging language-informed task representations for multi-task visual world models, enhancing multi-task learning in robotics.

Findings

Language-driven task representations improve world model performance.

Model-based multi-task learning outperforms model-free approaches.

Semantic task understanding benefits multi-task reinforcement learning.

Abstract

Most recent successes in robot reinforcement learning involve learning a specialized single-task agent. However, robots capable of performing multiple tasks can be much more valuable in real-world applications. Multi-task reinforcement learning can be very challenging due to the increased sample complexity and the potentially conflicting task objectives. Previous work on this topic is dominated by model-free approaches. The latter can be very sample inefficient even when learning specialized single-task agents. In this work, we focus on model-based multi-task reinforcement learning. We propose a method for learning multi-task visual world models, leveraging pre-trained language models to extract semantically meaningful task representations. These representations are used by the world model and policy to reason about task similarity in dynamics and behavior. Our results highlight the benefits of using language-driven task representations for world models and a clear advantage of model-based multi-task learning over the more common model-free paradigm.