Learning a Neural 3D Texture Space from 2D Exemplars

TL;DR

This paper introduces a generative model that creates diverse, high-fidelity 3D textures from 2D exemplars using a neural approach inspired by stochastic procedural texturing, enabling efficient and flexible texture synthesis.

Contribution

It presents a novel neural texture space model that encodes multiple exemplars without retraining, extending procedural texturing ideas to learned deep models for 3D textures.

Findings

Supports texture interpolation

Learns 3D textures from 2D exemplars

Operates efficiently without retraining for each exemplar

Abstract

We propose a generative model of 2D and 3D natural textures with diversity, visual fidelity and at high computational efficiency. This is enabled by a family of methods that extend ideas from classic stochastic procedural texturing (Perlin noise) to learned, deep, non-linearities. The key idea is a hard-coded, tunable and differentiable step that feeds multiple transformed random 2D or 3D fields into an MLP that can be sampled over infinite domains. Our model encodes all exemplars from a diverse set of textures without a need to be re-trained for each exemplar. Applications include texture interpolation, and learning 3D textures from 2D exemplars.