Simultaneous 3D Object Segmentation and 6-DOF Pose Estimation

TL;DR

This paper introduces a single-shot 3D point cloud method that simultaneously segments objects and estimates their 6-DOF poses, utilizing augmented reality for training data generation and a multi-task CNN architecture.

Contribution

It presents a novel, concise approach that avoids 2D projections and spatial transformations, leveraging augmented reality for training data in 3D object detection and pose estimation.

Findings

Achieves comparable or superior performance to state-of-the-art methods.

Effectively generalizes across multiple datasets and scenarios.

Utilizes augmented reality to generate training data in semi-virtual 3D space.

Abstract

We propose a single-shot method for simultaneous 3D object segmentation and 6-DOF pose estimation in pure 3D point clouds scenes based on a consensus that \emph{one point only belongs to one object}, i.e., each point has the potential power to predict the 6-DOF pose of its corresponding object. Unlike the recently proposed methods of the similar task, which rely on 2D detectors to predict the projection of 3D corners of the 3D bounding boxes and the 6-DOF pose must be estimated by a PnP like spatial transformation method, ours is concise enough not to require additional spatial transformation between different dimensions. Due to the lack of training data for many objects, the recently proposed 2D detection methods try to generate training data by using rendering engine and achieve good results. However, rendering in 3D space along with 6-DOF is relatively difficult. Therefore, we propose an augmented reality technology to generate the training data in semi-virtual reality 3D space. The key component of our method is a multi-task CNN architecture that can simultaneously predicts the 3D object segmentation and 6-DOF pose estimation in pure 3D point clouds. For experimental evaluation, we generate expanded training data for two state-of-the-arts 3D object datasets \cite{PLCHF}\cite{TLINEMOD} by using Augmented Reality technology (AR). We evaluate our proposed method on the two datasets. The results show that our method can be well generalized into multiple scenarios and provide performance comparable to or better than the state-of-the-arts.