Weakly-supervised 3D Shape Completion in the Wild

TL;DR

This paper introduces a weakly-supervised approach for 3D shape completion from real-world, unaligned partial point clouds, jointly estimating shape and pose to improve completion accuracy without extensive supervision.

Contribution

It proposes a novel method that learns 3D shape completion and pose estimation simultaneously from unaligned, real-world data, reducing the need for labeled supervision.

Findings

Effective on synthetic and real data

Enables shape completion from a single partial point cloud

Facilitates partial point cloud registration

Abstract

3D shape completion for real data is important but challenging, since partial point clouds acquired by real-world sensors are usually sparse, noisy and unaligned. Different from previous methods, we address the problem of learning 3D complete shape from unaligned and real-world partial point clouds. To this end, we propose a weakly-supervised method to estimate both 3D canonical shape and 6-DoF pose for alignment, given multiple partial observations associated with the same instance. The network jointly optimizes canonical shapes and poses with multi-view geometry constraints during training, and can infer the complete shape given a single partial point cloud. Moreover, learned pose estimation can facilitate partial point cloud registration. Experiments on both synthetic and real data show that it is feasible and promising to learn 3D shape completion through large-scale data without shape and pose supervision.