Robust Driving Policy Learning with Guided Meta Reinforcement Learning

TL;DR

This paper presents a meta reinforcement learning approach to train robust and diverse driving policies for social vehicles, improving autonomous navigation in unseen traffic scenarios by enhancing generalization to out-of-distribution behaviors.

Contribution

It introduces a method to train a single meta-policy for social vehicles with randomized rewards, and a strategy to improve the ego vehicle's robustness against diverse social behaviors.

Findings

Meta-policy effectively generates diverse social vehicle behaviors.

Ego policy generalizes well to unseen social agent behaviors.

Method outperforms fixed-policy approaches in complex scenarios.

Abstract

Although deep reinforcement learning (DRL) has shown promising results for autonomous navigation in interactive traffic scenarios, existing work typically adopts a fixed behavior policy to control social vehicles in the training environment. This may cause the learned driving policy to overfit the environment, making it difficult to interact well with vehicles with different, unseen behaviors. In this work, we introduce an efficient method to train diverse driving policies for social vehicles as a single meta-policy. By randomizing the interaction-based reward functions of social vehicles, we can generate diverse objectives and efficiently train the meta-policy through guiding policies that achieve specific objectives. We further propose a training strategy to enhance the robustness of the ego vehicle's driving policy using the environment where social vehicles are controlled by the learned meta-policy. Our method successfully learns an ego driving policy that generalizes well to unseen situations with out-of-distribution (OOD) social agents' behaviors in a challenging uncontrolled T-intersection scenario.