Combining Local and Global Pose Estimation for Precise Tracking of Similar Objects

TL;DR

This paper introduces a real-time multi-object 6D pose tracking system combining neural networks and local refinement, effective for similar, non-textured objects, with applications in augmented reality and construction.

Contribution

A novel multi-object pose estimation pipeline that integrates synthetic-trained neural networks with local geometric refinement for improved accuracy and efficiency.

Findings

Enhanced pose accuracy over baseline methods

Effective tracking of similar, non-textured objects

Real-time performance demonstrated in AR application

Abstract

In this paper, we present a multi-object 6D detection and tracking pipeline for potentially similar and non-textured objects. The combination of a convolutional neural network for object classification and rough pose estimation with a local pose refinement and an automatic mismatch detection enables direct application in real-time AR scenarios. A new network architecture, trained solely with synthetic images, allows simultaneous pose estimation of multiple objects with reduced GPU memory consumption and enhanced performance. In addition, the pose estimates are further improved by a local edge-based refinement step that explicitly exploits known object geometry information. For continuous movements, the sole use of local refinement reduces pose mismatches due to geometric ambiguities or occlusions. We showcase the entire tracking pipeline and demonstrate the benefits of the combined approach. Experiments on a challenging set of non-textured similar objects demonstrate the enhanced quality compared to the baseline method. Finally, we illustrate how the system can be used in a real AR assistance application within the field of construction.