What Matters to Enhance Traffic Rule Compliance of Imitation Learning for End-to-End Autonomous Driving

TL;DR

This paper introduces P-CSG, a penalty-based imitation learning method with cross semantics generation for end-to-end autonomous driving, improving traffic rule compliance, robustness, and performance in simulation benchmarks.

Contribution

The paper proposes a novel penalty-based imitation learning approach with cross semantics generation to enhance traffic rule adherence and robustness in end-to-end autonomous driving.

Findings

Achieved 8.5% and 2.0% improvements in driving scores on CARLA benchmarks.

Enhanced robustness against adversarial attacks like FGSM and Dot attacks.

Improved traffic rule sensitivity through specific penalties for red lights, stop signs, and curvature speed.

Abstract

End-to-end autonomous driving, where the entire driving pipeline is replaced with a single neural network, has recently gained research attention because of its simpler structure and faster inference time. Despite this appealing approach largely reducing the complexity in the driving pipeline, it also leads to safety issues because the trained policy is not always compliant with the traffic rules. In this paper, we proposed P-CSG, a penalty-based imitation learning approach with contrastive-based cross semantics generation sensor fusion technologies to increase the overall performance of end-to-end autonomous driving. In this method, we introduce three penalties - red light, stop sign, and curvature speed penalty to make the agent more sensitive to traffic rules. The proposed cross semantics generation helps to align the shared information of different input modalities. We assessed our model's performance using the CARLA Leaderboard - Town 05 Long Benchmark and Longest6 Benchmark, achieving 8.5% and 2.0% driving score improvement compared to the baselines. Furthermore, we conducted robustness evaluations against adversarial attacks like FGSM and Dot attacks, revealing a substantial increase in robustness compared to other baseline models. More detailed information can be found at https://hk-zh.github.io/p-csg-plus.