Exploring Imitation Learning for Autonomous Driving with Feedback Synthesizer and Differentiable Rasterization

TL;DR

This paper introduces a novel imitation learning framework for autonomous driving that incorporates a feedback synthesizer for data augmentation, a differentiable rasterizer for efficient training, and an attention mechanism for interpretability, resulting in improved safety and robustness.

Contribution

The work presents a new feedback synthesizer for data augmentation and a differentiable rasterizer to enhance imitation learning for autonomous driving, with improved efficiency and safety.

Findings

High driving performance in complex scenarios

Effective data augmentation improves robustness

Attention mechanism enhances interpretability

Abstract

We present a learning-based planner that aims to robustly drive a vehicle by mimicking human drivers' driving behavior. We leverage a mid-to-mid approach that allows us to manipulate the input to our imitation learning network freely. With that in mind, we propose a novel feedback synthesizer for data augmentation. It allows our agent to gain more driving experience in various previously unseen environments that are likely to encounter, thus improving overall performance. This is in contrast to prior works that rely purely on random synthesizers. Furthermore, rather than completely commit to imitating, we introduce task losses that penalize undesirable behaviors, such as collision, off-road, and so on. Unlike prior works, this is done by introducing a differentiable vehicle rasterizer that directly converts the waypoints output by the network into images. This effectively avoids the usage of heavyweight ConvLSTM networks, therefore, yields a faster model inference time. About the network architecture, we exploit an attention mechanism that allows the network to reason critical objects in the scene and produce better interpretable attention heatmaps. To further enhance the safety and robustness of the network, we add an optional optimization-based post-processing planner improving the driving comfort. We comprehensively validate our method's effectiveness in different scenarios that are specifically created for evaluating self-driving vehicles. Results demonstrate that our learning-based planner achieves high intelligence and can handle complex situations. Detailed ablation and visualization analysis are included to further demonstrate each of our proposed modules' effectiveness in our method.