3D Object Class Detection in the Wild

TL;DR

This paper introduces a novel 3D object class detection method that enhances 2D detection with viewpoint, keypoints, and 3D shape estimation, achieving state-of-the-art results on Pascal3D+.

Contribution

It presents a multi-stage 3D object detection framework that integrates viewpoint, keypoints, and shape estimation, advancing beyond traditional 2D detection methods.

Findings

Achieves state-of-the-art 2D bounding box and viewpoint estimation on Pascal3D+

Progressively improves detection accuracy through multi-stage design

Demonstrates effectiveness of 3D-aware detection in complex scenes

Abstract

Object class detection has been a synonym for 2D bounding box localization for the longest time, fueled by the success of powerful statistical learning techniques, combined with robust image representations. Only recently, there has been a growing interest in revisiting the promise of computer vision from the early days: to precisely delineate the contents of a visual scene, object by object, in 3D. In this paper, we draw from recent advances in object detection and 2D-3D object lifting in order to design an object class detector that is particularly tailored towards 3D object class detection. Our 3D object class detection method consists of several stages gradually enriching the object detection output with object viewpoint, keypoints and 3D shape estimates. Following careful design, in each stage it constantly improves the performance and achieves state-ofthe-art performance in simultaneous 2D bounding box and viewpoint estimation on the challenging Pascal3D+ dataset.