Reinforcement Learning based Control of Imitative Policies for Near-Accident Driving

TL;DR

This paper introduces a hierarchical reinforcement and imitation learning framework for autonomous driving in near-accident scenarios, enabling safer and more efficient decision-making by switching between specialized driving modes.

Contribution

The paper presents a novel hierarchical approach combining RL and IL to better handle high-risk driving situations with rapid state changes.

Findings

Higher safety and efficiency in near-accident scenarios

Effective switching between driving modes demonstrated

Improved policy performance over existing methods

Abstract

Autonomous driving has achieved significant progress in recent years, but autonomous cars are still unable to tackle high-risk situations where a potential accident is likely. In such near-accident scenarios, even a minor change in the vehicle's actions may result in drastically different consequences. To avoid unsafe actions in near-accident scenarios, we need to fully explore the environment. However, reinforcement learning (RL) and imitation learning (IL), two widely-used policy learning methods, cannot model rapid phase transitions and are not scalable to fully cover all the states. To address driving in near-accident scenarios, we propose a hierarchical reinforcement and imitation learning (H-ReIL) approach that consists of low-level policies learned by IL for discrete driving modes, and a high-level policy learned by RL that switches between different driving modes. Our approach exploits the advantages of both IL and RL by integrating them into a unified learning framework. Experimental results and user studies suggest our approach can achieve higher efficiency and safety compared to other methods. Analyses of the policies demonstrate our high-level policy appropriately switches between different low-level policies in near-accident driving situations.