Deep Multi-Task Learning for Joint Localization, Perception, and Prediction

TL;DR

This paper proposes a joint perception, prediction, and localization system for autonomous driving that efficiently corrects localization errors by sharing computation across tasks, demonstrated on large-scale datasets.

Contribution

It introduces a novel multi-task architecture that simultaneously performs perception, prediction, and localization to improve accuracy under localization errors.

Findings

Efficient correction of localization errors in autonomous systems

Improved perception and prediction accuracy with joint learning

Validated on large-scale autonomy dataset

Abstract

Over the last few years, we have witnessed tremendous progress on many subtasks of autonomous driving, including perception, motion forecasting, and motion planning. However, these systems often assume that the car is accurately localized against a high-definition map. In this paper we question this assumption, and investigate the issues that arise in state-of-the-art autonomy stacks under localization error. Based on our observations, we design a system that jointly performs perception, prediction, and localization. Our architecture is able to reuse computation between both tasks, and is thus able to correct localization errors efficiently. We show experiments on a large-scale autonomy dataset, demonstrating the efficiency and accuracy of our proposed approach.