One Noise to Rule Them All: Learning a Unified Model of Spatially-Varying Noise Patterns

TL;DR

This paper introduces a unified generative model for procedural noise that can produce and blend multiple noise types, including spatially-varying patterns, using a diffusion model trained with novel techniques, enhancing procedural graphics and material design.

Contribution

A single diffusion-based model capable of generating, blending, and spatially-varying multiple noise types, with controllability and improved inverse material reconstruction.

Findings

Model generates diverse noise textures.

Enables blending of different noise types.

Improves fidelity in inverse procedural material design.

Abstract

Procedural noise is a fundamental component of computer graphics pipelines, offering a flexible way to generate textures that exhibit "natural" random variation. Many different types of noise exist, each produced by a separate algorithm. In this paper, we present a single generative model which can learn to generate multiple types of noise as well as blend between them. In addition, it is capable of producing spatially-varying noise blends despite not having access to such data for training. These features are enabled by training a denoising diffusion model using a novel combination of data augmentation and network conditioning techniques. Like procedural noise generators, the model's behavior is controllable via interpretable parameters and a source of randomness. We use our model to produce a variety of visually compelling noise textures. We also present an application of our model to improving inverse procedural material design; using our model in place of fixed-type noise nodes in a procedural material graph results in higher-fidelity material reconstructions without needing to know the type of noise in advance.