Learning Accurate and Human-Like Driving using Semantic Maps and Attention

TL;DR

This paper presents an end-to-end driving model that leverages semantic maps and attention mechanisms to improve accuracy and human-likeness, trained on extensive real-world driving data.

Contribution

It introduces a novel attention mechanism using semantic maps and employs adversarial learning to enhance the human-likeness of driving behavior.

Findings

Models outperform previous methods in accuracy.

Driving behavior is more human-like.

Utilizes extensive real-world driving dataset.

Abstract

This paper investigates how end-to-end driving models can be improved to drive more accurately and human-like. To tackle the first issue we exploit semantic and visual maps from HERE Technologies and augment the existing Drive360 dataset with such. The maps are used in an attention mechanism that promotes segmentation confidence masks, thus focusing the network on semantic classes in the image that are important for the current driving situation. Human-like driving is achieved using adversarial learning, by not only minimizing the imitation loss with respect to the human driver but by further defining a discriminator, that forces the driving model to produce action sequences that are human-like. Our models are trained and evaluated on the Drive360 + HERE dataset, which features 60 hours and 3000 km of real-world driving data. Extensive experiments show that our driving models are more accurate and behave more human-like than previous methods.