Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactions

TL;DR

This paper introduces a convolutional model that predicts future driving behaviors by leveraging high-level semantic maps and 3D perception data, enabling more accurate and context-aware predictions in complex driving scenarios.

Contribution

It presents a unified semantic grid representation and a deep convolutional approach to model interactions, along with a new dataset for training and evaluation.

Findings

Effective learning of driving behavior fundamentals

Utilization of rich semantic and perception data improves prediction accuracy

Model can predict distributions over future states

Abstract

We focus on the problem of predicting future states of entities in complex, real-world driving scenarios. Previous research has used low-level signals to predict short time horizons, and has not addressed how to leverage key assets relied upon heavily by industry self-driving systems: (1) large 3D perception efforts which provide highly accurate 3D states of agents with rich attributes, and (2) detailed and accurate semantic maps of the environment (lanes, traffic lights, crosswalks, etc). We present a unified representation which encodes such high-level semantic information in a spatial grid, allowing the use of deep convolutional models to fuse complex scene context. This enables learning entity-entity and entity-environment interactions with simple, feed-forward computations in each timestep within an overall temporal model of an agent's behavior. We propose different ways of modelling the future as a distribution over future states using standard supervised learning. We introduce a novel dataset providing industry-grade rich perception and semantic inputs, and empirically show we can effectively learn fundamentals of driving behavior.