YOLO3D: End-to-end real-time 3D Oriented Object Bounding Box Detection from LiDAR Point Cloud

TL;DR

This paper introduces YOLO3D, a real-time 3D object detection method from LiDAR data that extends YOLO v2 to include 3D bounding box parameters, achieving 40 fps on a Titan X GPU.

Contribution

It extends YOLO v2 with a new loss function for 3D bounding box parameters, enabling real-time 3D object detection from LiDAR point clouds.

Findings

Achieves 40 fps on Titan X GPU

Performs well on KITTI benchmark

Extends YOLO v2 for 3D detection

Abstract

Object detection and classification in 3D is a key task in Automated Driving (AD). LiDAR sensors are employed to provide the 3D point cloud reconstruction of the surrounding environment, while the task of 3D object bounding box detection in real time remains a strong algorithmic challenge. In this paper, we build on the success of the one-shot regression meta-architecture in the 2D perspective image space and extend it to generate oriented 3D object bounding boxes from LiDAR point cloud. Our main contribution is in extending the loss function of YOLO v2 to include the yaw angle, the 3D box center in Cartesian coordinates and the height of the box as a direct regression problem. This formulation enables real-time performance, which is essential for automated driving. Our results are showing promising figures on KITTI benchmark, achieving real-time performance (40 fps) on Titan X GPU.