Automated Driving with Evolution Capability: A Reinforcement Learning Method with Monotonic Performance Enhancement

TL;DR

This paper introduces HCPI-RL, a reinforcement learning method for automated driving that guarantees monotonic performance improvement, handles emergency scenarios, and significantly enhances decision-making and efficiency over existing planners.

Contribution

The paper proposes a novel HCPI-RL framework that ensures consistent performance improvement in automated driving, addressing risks associated with traditional RL methods.

Findings

Policy return increased by up to 108.2% in emergency scenarios

Driving efficiency improved by 19.2% over PPO planner

Monotonic performance enhancement achieved in diverse scenarios

Abstract

Reinforcement Learning (RL) offers a promising solution to enable evolutionary automated driving. However, the conventional RL method is always concerned with risk performance. The updated policy may not obtain a performance enhancement, even leading to performance deterioration. To address this challenge, this research proposes a High Confidence Policy Improvement Reinforcement Learning-based (HCPI-RL) planner. It is intended to achieve the monotonic evolution of automated driving. A novel RL policy update paradigm is designed to enable the newly learned policy performance consistently surpass that of previous policies, which is deemed as monotonic performance enhancement. Hence, the proposed HCPI-RL planner has the following features: i) Evolutionary automated driving with monotonic performance enhancement; ii) With the capability of handling scenarios with emergency; iii) With enhanced decision-making optimality. Results demonstrate that the proposed HCPI-RL planner enhances the policy return by 44.7% in emergent cut-in scenarios, 108.2% in emergent braking scenarios, and 64.4% in daily cruising scenarios, compared to the PPO planner. Adopting the proposed planner, automated driving efficiency is enhanced by 19.2% compared to the PPO planner, and by 30.7% compared to the rule-based planner.