SEMPose: A Single End-to-end Network for Multi-object Pose Estimation

TL;DR

SEMPose is an end-to-end neural network that accurately estimates multiple object poses from RGB images in real time, effectively handling size variations and occlusions without additional inputs.

Contribution

The paper introduces SEMPose, a novel end-to-end multi-object pose estimation network with a texture-shape guided feature pyramid and iterative refinement, improving accuracy and speed over existing methods.

Findings

Achieves 32 FPS inference speed on standard datasets.

Outperforms existing RGB-based single-model methods in accuracy.

Maintains consistent inference time regardless of object count.

Abstract

In computer vision, estimating the six-degree-of-freedom pose from an RGB image is a fundamental task. However, this task becomes highly challenging in multi-object scenes. Currently, the best methods typically employ an indirect strategy, which identifies 2D and 3D correspondences, and then solves with the Perspective-n-Points method. Yet, this approach cannot be trained end-to-end. Direct methods, on the other hand, suffer from lower accuracy due to challenges such as varying object sizes and occlusions. To address these issues, we propose SEMPose, an end-to-end multi-object pose estimation network. SEMPose utilizes a well-designed texture-shape guided feature pyramid network, effectively tackling the challenge of object size variations. Additionally, it employs an iterative refinement head structure, progressively regressing rotation and translation separately to enhance estimation accuracy. During training, we alleviate the impact of occlusion by selecting positive samples from visible parts. Experimental results demonstrate that SEMPose can perform inference at 32 FPS without requiring inputs other than the RGB image. It can accurately estimate the poses of multiple objects in real time, with inference time unaffected by the number of target objects. On the LM-O and YCB-V datasets, our method outperforms other RGB-based single-model methods, achieving higher accuracy. Even when compared with multi-model methods and approaches that use additional refinement, our results remain competitive.