Dream to Drive with Predictive Individual World Model

TL;DR

This paper introduces a novel model-based reinforcement learning approach with a predictive individual world model for autonomous driving, effectively capturing vehicle interactions and intentions to improve safety and efficiency in urban environments.

Contribution

It presents a new PIWM that models vehicle interactions and intentions at an individual level, enhancing scene understanding and decision-making in autonomous driving.

Findings

Outperforms state-of-the-art methods in safety and efficiency

Effectively models vehicle interactions and intentions

Achieves superior performance in simulation environments

Abstract

It is still a challenging topic to make reactive driving behaviors in complex urban environments as road users' intentions are unknown. Model-based reinforcement learning (MBRL) offers great potential to learn a reactive policy by constructing a world model that can provide informative states and imagination training. However, a critical limitation in relevant research lies in the scene-level reconstruction representation learning, which may overlook key interactive vehicles and hardly model the interactive features among vehicles and their long-term intentions. Therefore, this paper presents a novel MBRL method with a predictive individual world model (PIWM) for autonomous driving. PIWM describes the driving environment from an individual-level perspective and captures vehicles' interactive relations and their intentions via trajectory prediction task. Meanwhile, a behavior policy is learned jointly with PIWM. It is trained in PIWM's imagination and effectively navigates in the urban driving scenes leveraging intention-aware latent states. The proposed method is trained and evaluated on simulation environments built upon real-world challenging interactive scenarios. Compared with popular model-free and state-of-the-art model-based reinforcement learning methods, experimental results show that the proposed method achieves the best performance in terms of safety and efficiency.