Robust Category-Level 6D Pose Estimation with Coarse-to-Fine Rendering of Neural Features

TL;DR

This paper introduces a novel coarse-to-fine rendering approach using neural features for robust category-level 6D pose estimation from a single RGB image, effectively handling occlusions and improving accuracy.

Contribution

It proposes a scale- and rotation-invariant neural feature representation combined with a coarse-to-fine optimization strategy for pose estimation.

Findings

Improved pose estimation accuracy over prior methods.

Enhanced robustness under partial occlusion.

Effective use of differentiable rendering with neural features.

Abstract

We consider the problem of category-level 6D pose estimation from a single RGB image. Our approach represents an object category as a cuboid mesh and learns a generative model of the neural feature activations at each mesh vertex to perform pose estimation through differentiable rendering. A common problem of rendering-based approaches is that they rely on bounding box proposals, which do not convey information about the 3D rotation of the object and are not reliable when objects are partially occluded. Instead, we introduce a coarse-to-fine optimization strategy that utilizes the rendering process to estimate a sparse set of 6D object proposals, which are subsequently refined with gradient-based optimization. The key to enabling the convergence of our approach is a neural feature representation that is trained to be scale- and rotation-invariant using contrastive learning. Our experiments demonstrate an enhanced category-level 6D pose estimation performance compared to prior work, particularly under strong partial occlusion.