Polarimetric Information for Multi-Modal 6D Pose Estimation of Photometrically Challenging Objects with Limited Data

TL;DR

This paper introduces a supervised and self-supervised polarimetric approach for 6D pose estimation of challenging objects, overcoming limitations of traditional RGB-based methods by leveraging physical properties of polarized light.

Contribution

It presents a novel supervised and self-supervised method that uses polarimetric data and physical constraints to improve pose estimation of difficult objects.

Findings

Significant improvement in pose accuracy for photometrically challenging objects.

Effective self-supervised training without annotated real data.

Enhanced geometric understanding through polarimetric information.

Abstract

6D pose estimation pipelines that rely on RGB-only or RGB-D data show limitations for photometrically challenging objects with e.g. textureless surfaces, reflections or transparency. A supervised learning-based method utilising complementary polarisation information as input modality is proposed to overcome such limitations. This supervised approach is then extended to a self-supervised paradigm by leveraging physical characteristics of polarised light, thus eliminating the need for annotated real data. The methods achieve significant advancements in pose estimation by leveraging geometric information from polarised light and incorporating shape priors and invertible physical constraints.