Hindsight is 20/20: Leveraging Past Traversals to Aid 3D Perception

TL;DR

This paper introduces a novel framework that utilizes past traversal data to enhance 3D object detection in autonomous driving, significantly improving detection accuracy for challenging cases.

Contribution

The paper presents an end-to-end trainable Hindsight framework that leverages past scene data to improve 3D perception in autonomous vehicles, a novel approach in the field.

Findings

Substantially improves detection accuracy on multiple datasets.

Achieves over 300% improvement on challenging detection cases.

Compatible with most modern 3D detection architectures.

Abstract

Self-driving cars must detect vehicles, pedestrians, and other traffic participants accurately to operate safely. Small, far-away, or highly occluded objects are particularly challenging because there is limited information in the LiDAR point clouds for detecting them. To address this challenge, we leverage valuable information from the past: in particular, data collected in past traversals of the same scene. We posit that these past data, which are typically discarded, provide rich contextual information for disambiguating the above-mentioned challenging cases. To this end, we propose a novel, end-to-end trainable Hindsight framework to extract this contextual information from past traversals and store it in an easy-to-query data structure, which can then be leveraged to aid future 3D object detection of the same scene. We show that this framework is compatible with most modern 3D detection architectures and can substantially improve their average precision on multiple autonomous driving datasets, most notably by more than 300% on the challenging cases.