Unsupervised Shape Completion via Deep Prior in the Neural Tangent Kernel Perspective

TL;DR

This paper introduces an unsupervised 3D shape completion method using deep neural networks optimized per shape, interpreted through the neural tangent kernel, enabling plausible reconstructions without prior training data.

Contribution

It offers a novel NTK-based interpretation of deep priors for shape completion, leading to more efficient networks and adaptable, unsupervised reconstruction of incomplete 3D shapes.

Findings

Successfully completes large missing regions with plausible shapes.

Does not require prior training data, enabling flexible shape adaptation.

Outperforms traditional methods in structural regularity awareness.

Abstract

We present a novel approach for completing and reconstructing 3D shapes from incomplete scanned data by using deep neural networks. Rather than being trained on supervised completion tasks and applied on a testing shape, the network is optimized from scratch on the single testing shape, to fully adapt to the shape and complete the missing data using contextual guidance from the known regions. The ability to complete missing data by an untrained neural network is usually referred to as the deep prior. In this paper, we interpret the deep prior from a neural tangent kernel (NTK) perspective and show that the completed shape patches by the trained CNN are naturally similar to existing patches, as they are proximate in the kernel feature space induced by NTK. The interpretation allows us to design more efficient network structures and learning mechanisms for the shape completion and reconstruction task. Being more aware of structural regularities than both traditional and other unsupervised learning-based reconstruction methods, our approach completes large missing regions with plausible shapes and complements supervised learning-based methods that use database priors by requiring no extra training data set and showing flexible adaptation to a particular shape instance.