Improve Generalization of Driving Policy at Signalized Intersections with Adversarial Learning

TL;DR

This paper presents an adversarial learning framework to improve the robustness and safety of autonomous driving policies at signalized intersections, effectively handling environmental uncertainties and diverse traffic behaviors.

Contribution

It introduces a novel adversarial policy gradient method combined with a static path planner and optimal control to enhance driving policy robustness at complex intersections.

Findings

Enhanced policy robustness against uncertainties

Improved safety and efficiency in intersection navigation

Large-margin resistance to abnormal behaviors

Abstract

Intersections are quite challenging among various driving scenes wherein the interaction of signal lights and distinct traffic actors poses great difficulty to learn a wise and robust driving policy. Current research rarely considers the diversity of intersections and stochastic behaviors of traffic participants. For practical applications, the randomness usually leads to some devastating events, which should be the focus of autonomous driving. This paper introduces an adversarial learning paradigm to boost the intelligence and robustness of driving policy for signalized intersections with dense traffic flow. Firstly, we design a static path planner which is capable of generating trackable candidate paths for multiple intersections with diversified topology. Next, a constrained optimal control problem (COCP) is built based on these candidate paths wherein the bounded uncertainty of dynamic models is considered to capture the randomness of driving environment. We propose adversarial policy gradient (APG) to solve the COCP wherein the adversarial policy is introduced to provide disturbances by seeking the most severe uncertainty while the driving policy learns to handle this situation by competition. Finally, a comprehensive system is established to conduct training and testing wherein the perception module is introduced and the human experience is incorporated to solve the yellow light dilemma. Experiments indicate that the trained policy can handle the signal lights flexibly meanwhile realizing the smooth and efficient passing with a humanoid paradigm. Besides, APG enables a large-margin improvement of the resistance to the abnormal behaviors and thus ensures a high safety level for the autonomous vehicle.