DRACO: Weakly Supervised Dense Reconstruction And Canonicalization of Objects

TL;DR

DRACO is a novel method that achieves dense 3D object shape reconstruction and canonicalization from RGB images using only weak supervision, outperforming some fully-supervised approaches.

Contribution

It introduces a weakly supervised approach for dense canonical shape reconstruction from RGB images, reducing reliance on dense 3D supervision.

Findings

DRACO performs comparably or better than fully-supervised methods.

It successfully reconstructs dense object shapes in a canonical coordinate space.

The method works effectively with only camera poses and semantic keypoints during training.

Abstract

We present DRACO, a method for Dense Reconstruction And Canonicalization of Object shape from one or more RGB images. Canonical shape reconstruction, estimating 3D object shape in a coordinate space canonicalized for scale, rotation, and translation parameters, is an emerging paradigm that holds promise for a multitude of robotic applications. Prior approaches either rely on painstakingly gathered dense 3D supervision, or produce only sparse canonical representations, limiting real-world applicability. DRACO performs dense canonicalization using only weak supervision in the form of camera poses and semantic keypoints at train time. During inference, DRACO predicts dense object-centric depth maps in a canonical coordinate-space, solely using one or more RGB images of an object. Extensive experiments on canonical shape reconstruction and pose estimation show that DRACO is competitive or superior to fully-supervised methods.