Implicit Feature Networks for Texture Completion from Partial 3D Data

TL;DR

This paper introduces a novel method for completing 3D textures on partial scans of objects and humans by leveraging deep features from both texture and geometry, improving over previous approaches that lacked texture output.

Contribution

We extend Implicit Feature Networks to perform 3D texture completion, integrating local and global features from partial textures and geometries for more accurate in-painting.

Findings

Achieved state-of-the-art results on 3D texture completion tasks.

Won the SHARP ECCV'20 challenge with highest performance.

Demonstrated effective in-painting of missing textures in complex 3D scans.

Abstract

Prior work to infer 3D texture use either texture atlases, which require uv-mappings and hence have discontinuities, or colored voxels, which are memory inefficient and limited in resolution. Recent work, predicts RGB color at every XYZ coordinate forming a texture field, but focus on completing texture given a single 2D image. Instead, we focus on 3D texture and geometry completion from partial and incomplete 3D scans. IF-Nets have recently achieved state-of-the-art results on 3D geometry completion using a multi-scale deep feature encoding, but the outputs lack texture. In this work, we generalize IF-Nets to texture completion from partial textured scans of humans and arbitrary objects. Our key insight is that 3D texture completion benefits from incorporating local and global deep features extracted from both the 3D partial texture and completed geometry. Specifically, given the partial 3D texture and the 3D geometry completed with IF-Nets, our model successfully in-paints the missing texture parts in consistence with the completed geometry. Our model won the SHARP ECCV'20 challenge, achieving highest performance on all challenges.