Cov2Pose: Leveraging Spatial Covariance for Direct Manifold-aware 6-DoF Object Pose Estimation

TL;DR

This paper introduces Cov2Pose, a novel method for 6-DoF object pose estimation from a single RGB image that leverages spatial covariance and manifold-aware regression to improve accuracy and robustness over traditional direct methods.

Contribution

It proposes a covariance-pooled representation and a manifold-aware network head for direct pose regression, incorporating second-order statistics and continuous pose encoding.

Findings

Improved accuracy in pose estimation, especially under partial occlusion.

Demonstrates the effectiveness of second-order pooling and SPD matrix representations.

Outperforms existing direct regression methods in experiments.

Abstract

In this paper, we address the problem of 6-DoF object pose estimation from a single RGB image. Indirect methods that typically predict intermediate 2D keypoints, followed by a Perspective-n-Point solver, have shown great performance. Direct approaches, which regress the pose in an end-to-end manner, are usually computationally more efficient but less accurate. However, direct pose regression heads rely on globally pooled features, ignoring spatial second-order statistics despite their informativeness in pose prediction. They also predict, in most cases, discontinuous pose representations that lack robustness. Herein, we therefore propose a covariance-pooled representation that encodes convolutional feature distributions as a symmetric positive definite (SPD) matrix. Moreover, we propose a novel pose encoding in the form of an SPD matrix via its Cholesky decomposition. Pose is then regressed in an end-to-end manner with a manifold-aware network head, taking into account the Riemannian geometry of SPD matrices. Experiments and ablations consistently demonstrate the relevance of second-order pooling and continuous representations for direct pose regression, including under partial occlusion.