Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racing

TL;DR

This paper demonstrates that world models with latent imagination significantly improve zero-shot transfer and generalization in autonomous racing robots, especially in complex real-world tasks, outperforming model-free approaches.

Contribution

It introduces a novel application of latent imagination world models to real-world autonomous vehicle control, showing their effectiveness in sim2real transfer and generalization.

Findings

Model-based agents outperform model-free agents in performance and sample efficiency.

Longer memory horizons in world models enhance sim2real transfer.

Observation model choice critically affects generalization ability.

Abstract

World models learn behaviors in a latent imagination space to enhance the sample-efficiency of deep reinforcement learning (RL) algorithms. While learning world models for high-dimensional observations (e.g., pixel inputs) has become practicable on standard RL benchmarks and some games, their effectiveness in real-world robotics applications has not been explored. In this paper, we investigate how such agents generalize to real-world autonomous vehicle control tasks, where advanced model-free deep RL algorithms fail. In particular, we set up a series of time-lap tasks for an F1TENTH racing robot, equipped with a high-dimensional LiDAR sensor, on a set of test tracks with a gradual increase in their complexity. In this continuous-control setting, we show that model-based agents capable of learning in imagination substantially outperform model-free agents with respect to performance, sample efficiency, successful task completion, and generalization. Moreover, we show that the generalization ability of model-based agents strongly depends on the choice of their observation model. We provide extensive empirical evidence for the effectiveness of world models provided with long enough memory horizons in sim2real tasks.