Multimodal Shape Completion via Conditional Generative Adversarial Networks

TL;DR

This paper introduces a novel multimodal shape completion method using conditional generative adversarial networks that produces diverse and high-quality completions for partial 3D shapes without needing paired training data.

Contribution

It is the first to address multimodal shape completion with a conditional generative model that captures multiple plausible completions without paired datasets.

Findings

Outperforms baseline methods in diversity and quality of shape completions

Effective across various datasets with different types of shape incompleteness

Demonstrates the ability to generate multiple plausible shape completions

Abstract

Several deep learning methods have been proposed for completing partial data from shape acquisition setups, i.e., filling the regions that were missing in the shape. These methods, however, only complete the partial shape with a single output, ignoring the ambiguity when reasoning the missing geometry. Hence, we pose a multi-modal shape completion problem, in which we seek to complete the partial shape with multiple outputs by learning a one-to-many mapping. We develop the first multimodal shape completion method that completes the partial shape via conditional generative modeling, without requiring paired training data. Our approach distills the ambiguity by conditioning the completion on a learned multimodal distribution of possible results. We extensively evaluate the approach on several datasets that contain varying forms of shape incompleteness, and compare among several baseline methods and variants of our methods qualitatively and quantitatively, demonstrating the merit of our method in completing partial shapes with both diversity and quality.