SDFEst: Categorical Pose and Shape Estimation of Objects from RGB-D using Signed Distance Fields

TL;DR

SDFEst is a modular pipeline that estimates object pose and shape from RGB-D images using signed distance fields, integrating a generative shape model, a novel initialization network, and a differentiable renderer for efficient analysis-by-synthesis optimization.

Contribution

We introduce a novel approach combining signed distance fields with a modular framework for accurate pose and shape estimation from RGB-D data.

Findings

Outperforms state-of-the-art methods on synthetic and real datasets.

Enables multi-view optimization for improved accuracy.

Provides an open-source implementation for the community.

Abstract

Rich geometric understanding of the world is an important component of many robotic applications such as planning and manipulation. In this paper, we present a modular pipeline for pose and shape estimation of objects from RGB-D images given their category. The core of our method is a generative shape model, which we integrate with a novel initialization network and a differentiable renderer to enable 6D pose and shape estimation from a single or multiple views. We investigate the use of discretized signed distance fields as an efficient shape representation for fast analysis-by-synthesis optimization. Our modular framework enables multi-view optimization and extensibility. We demonstrate the benefits of our approach over state-of-the-art methods in several experiments on both synthetic and real data. We open-source our approach at https://github.com/roym899/sdfest.