Texture Fields: Learning Texture Representations in Function Space

TL;DR

This paper introduces Texture Fields, a neural network-based continuous 3D texture representation that improves high-frequency texture modeling and integrates seamlessly with deep learning for 3D object texturing.

Contribution

The paper proposes Texture Fields, a novel shape-independent texture representation that overcomes limitations of existing methods and enhances generative modeling of 3D textures.

Findings

Outperforms state-of-the-art methods in conditional texture reconstruction

Enables probabilistic generative modeling of unseen 3D textures

Effectively represents high-frequency textures

Abstract

In recent years, substantial progress has been achieved in learning-based reconstruction of 3D objects. At the same time, generative models were proposed that can generate highly realistic images. However, despite this success in these closely related tasks, texture reconstruction of 3D objects has received little attention from the research community and state-of-the-art methods are either limited to comparably low resolution or constrained experimental setups. A major reason for these limitations is that common representations of texture are inefficient or hard to interface for modern deep learning techniques. In this paper, we propose Texture Fields, a novel texture representation which is based on regressing a continuous 3D function parameterized with a neural network. Our approach circumvents limiting factors like shape discretization and parameterization, as the proposed texture representation is independent of the shape representation of the 3D object. We show that Texture Fields are able to represent high frequency texture and naturally blend with modern deep learning techniques. Experimentally, we find that Texture Fields compare favorably to state-of-the-art methods for conditional texture reconstruction of 3D objects and enable learning of probabilistic generative models for texturing unseen 3D models. We believe that Texture Fields will become an important building block for the next generation of generative 3D models.