Symmetry Aware Evaluation of 3D Object Detection and Pose Estimation in Scenes of Many Parts in Bulk

TL;DR

This paper introduces a new dataset and evaluation method for 3D object detection and pose estimation that accounts for object symmetries, enabling more accurate assessment in complex scenes with many parts.

Contribution

It provides an automatic annotation technique for large-scale RGBD data and a symmetry-aware evaluation protocol for rigid object pose estimation.

Findings

Symmetry-aware evaluation improves pose estimation accuracy.

The dataset includes thousands of scenes with real and synthetic images.

Incorporating symmetry into methods yields significant performance gains.

Abstract

While 3D object detection and pose estimation has been studied for a long time, its evaluation is not yet completely satisfactory. Indeed, existing datasets typically consist in numerous acquisitions of only a few scenes because of the tediousness of pose annotation, and existing evaluation protocols cannot handle properly objects with symmetries. This work aims at addressing those two points. We first present automatic techniques to produce fully annotated RGBD data of many object instances in arbitrary poses, with which we produce a dataset of thousands of independent scenes of bulk parts composed of both real and synthetic images. We then propose a consistent evaluation methodology suitable for any rigid object, regardless of its symmetries. We illustrate it with two reference object detection and pose estimation methods on different objects, and show that incorporating symmetry considerations into pose estimation methods themselves can lead to significant performance gains. The proposed dataset is available at http://rbregier.github.io/dataset2017.