Joint 3D Proposal Generation and Object Detection from View Aggregation

TL;DR

AVOD is a neural network architecture that combines LIDAR and RGB data to generate 3D object proposals and detections in real-time, achieving state-of-the-art results for autonomous driving applications.

Contribution

The paper introduces a novel multimodal fusion architecture for 3D object detection that operates efficiently in real-time and improves detection accuracy on the KITTI benchmark.

Findings

State-of-the-art performance on KITTI 3D detection benchmark

Real-time processing capability

Low memory footprint

Abstract

We present AVOD, an Aggregate View Object Detection network for autonomous driving scenarios. The proposed neural network architecture uses LIDAR point clouds and RGB images to generate features that are shared by two subnetworks: a region proposal network (RPN) and a second stage detector network. The proposed RPN uses a novel architecture capable of performing multimodal feature fusion on high resolution feature maps to generate reliable 3D object proposals for multiple object classes in road scenes. Using these proposals, the second stage detection network performs accurate oriented 3D bounding box regression and category classification to predict the extents, orientation, and classification of objects in 3D space. Our proposed architecture is shown to produce state of the art results on the KITTI 3D object detection benchmark while running in real time with a low memory footprint, making it a suitable candidate for deployment on autonomous vehicles. Code is at: https://github.com/kujason/avod