Faraway-Frustum: Dealing with Lidar Sparsity for 3D Object Detection using Fusion

TL;DR

This paper introduces Faraway-Frustum, a fusion method that improves 3D object detection at long distances by relying on 2D vision for recognition and pointcloud data for localization, outperforming existing methods.

Contribution

The paper proposes a novel fusion strategy that combines 2D vision and pointcloud data differently based on object distance, enhancing faraway object detection in autonomous driving.

Findings

Outperforms state-of-the-art in faraway object detection on KITTI dataset

Effective fusion strategy for different object distances

Significant improvement in 3D and bird's-eye-view detection accuracy

Abstract

Learned pointcloud representations do not generalize well with an increase in distance to the sensor. For example, at a range greater than 60 meters, the sparsity of lidar pointclouds reaches to a point where even humans cannot discern object shapes from each other. However, this distance should not be considered very far for fast-moving vehicles: A vehicle can traverse 60 meters under two seconds while moving at 70 mph. For safe and robust driving automation, acute 3D object detection at these ranges is indispensable. Against this backdrop, we introduce faraway-frustum: a novel fusion strategy for detecting faraway objects. The main strategy is to depend solely on the 2D vision for recognizing object class, as object shape does not change drastically with an increase in depth, and use pointcloud data for object localization in the 3D space for faraway objects. For closer objects, we use learned pointcloud representations instead, following state-of-the-art. This strategy alleviates the main shortcoming of object detection with learned pointcloud representations. Experiments on the KITTI dataset demonstrate that our method outperforms state-of-the-art by a considerable margin for faraway object detection in bird's-eye-view and 3D. Our code is open-source and publicly available: https://github.com/dongfang-steven-yang/faraway-frustum.