For A More Comprehensive Evaluation of 6DoF Object Pose Tracking

TL;DR

This paper introduces a unified benchmark for 6DoF object pose tracking, addressing previous evaluation limitations by proposing a new pose refinement method and improved metrics, enabling fairer comparisons across methods.

Contribution

It presents a comprehensive benchmark with a novel multi-view pose refinement technique and improved evaluation metrics for 6DoF pose tracking.

Findings

Refined YCBV annotations with sub-pixel accuracy

Validated the pose refinement method on a semi-synthesized dataset

Unified evaluation of learning and non-learning, RGB and RGBD methods

Abstract

Previous evaluations on 6DoF object pose tracking have presented obvious limitations along with the development of this area. In particular, the evaluation protocols are not unified for different methods, the widely-used YCBV dataset contains significant annotation error, and the error metrics also may be biased. As a result, it is hard to fairly compare the methods, which has became a big obstacle for developing new algorithms. In this paper we contribute a unified benchmark to address the above problems. For more accurate annotation of YCBV, we propose a multi-view multi-object global pose refinement method, which can jointly refine the poses of all objects and view cameras, resulting in sub-pixel sub-millimeter alignment errors. The limitations of previous scoring methods and error metrics are analyzed, based on which we introduce our improved evaluation methods. The unified benchmark takes both YCBV and BCOT as base datasets, which are shown to be complementary in scene categories. In experiments, we validate the precision and reliability of the proposed global pose refinement method with a realistic semi-synthesized dataset particularly for YCBV, and then present the benchmark results unifying learning&non-learning and RGB&RGBD methods, with some finds not discovered in previous studies.