High-Resolution Shape Completion Using Deep Neural Networks for Global Structure and Local Geometry Inference

TL;DR

This paper introduces a deep learning approach for 3D shape completion that combines global structure inference with local geometry refinement, achieving high-resolution, accurate reconstructions.

Contribution

It presents a novel two-network architecture with end-to-end training for improved 3D shape completion, integrating multi-view and volumetric information.

Findings

Outperforms state-of-the-art methods on six object categories

Produces high-resolution, complete 3D shapes

Effectively combines global and local shape information

Abstract

We propose a data-driven method for recovering miss-ing parts of 3D shapes. Our method is based on a new deep learning architecture consisting of two sub-networks: a global structure inference network and a local geometry refinement network. The global structure inference network incorporates a long short-term memorized context fusion module (LSTM-CF) that infers the global structure of the shape based on multi-view depth information provided as part of the input. It also includes a 3D fully convolutional (3DFCN) module that further enriches the global structure representation according to volumetric information in the input. Under the guidance of the global structure network, the local geometry refinement network takes as input lo-cal 3D patches around missing regions, and progressively produces a high-resolution, complete surface through a volumetric encoder-decoder architecture. Our method jointly trains the global structure inference and local geometry refinement networks in an end-to-end manner. We perform qualitative and quantitative evaluations on six object categories, demonstrating that our method outperforms existing state-of-the-art work on shape completion.