3D-FFS: Faster 3D object detection with Focused Frustum Search in sensor fusion based networks

TL;DR

The paper introduces 3D-FFS, a method that accelerates 3D object detection in sensor fusion networks by using heuristics based on point cloud properties, significantly reducing computational costs while maintaining accuracy.

Contribution

It presents 3D-FFS, a novel heuristic approach that constrains the 3D search space in sensor fusion networks, improving speed and efficiency without sacrificing detection accuracy.

Findings

Training time reduced by up to 62.80%

Inference time reduced by up to 58.96%

Memory usage decreased by up to 58.53%

Abstract

In this work we propose 3D-FFS, a novel approach to make sensor fusion based 3D object detection networks significantly faster using a class of computationally inexpensive heuristics. Existing sensor fusion based networks generate 3D region proposals by leveraging inferences from 2D object detectors. However, as images have no depth information, these networks rely on extracting semantic features of points from the entire scene to locate the object. By leveraging aggregated intrinsic properties (e.g. point density) of point cloud data, 3D-FFS can substantially constrain the 3D search space and thereby significantly reduce training time, inference time and memory consumption without sacrificing accuracy. To demonstrate the efficacy of 3D-FFS, we have integrated it with Frustum ConvNet (F-ConvNet), a prominent sensor fusion based 3D object detection model. We assess the performance of 3D-FFS on the KITTI dataset. Compared to F-ConvNet, we achieve improvements in training and inference times by up to 62.80% and 58.96%, respectively, while reducing the memory usage by up to 58.53%. Additionally, we achieve 0.36%, 0.59% and 2.19% improvements in accuracy for the Car, Pedestrian and Cyclist classes, respectively. 3D-FFS shows a lot of promise in domains with limited computing power, such as autonomous vehicles, drones and robotics where LiDAR-Camera based sensor fusion perception systems are widely used.