Infinite Texture: Text-guided High Resolution Diffusion Texture Synthesis

TL;DR

Infinite Texture is a novel method that fine-tunes a diffusion model on a single texture to generate arbitrarily large, high-resolution textures from text prompts, enabling scalable and customizable texture synthesis.

Contribution

The paper introduces a fine-tuning approach for diffusion models on a single texture, allowing high-resolution, large-scale texture generation guided by text prompts.

Findings

Produces high-quality, arbitrarily large textures from a single sample

Outperforms existing patch-based and deep learning texture synthesis methods

Enables applications in 3D rendering and texture transfer

Abstract

We present Infinite Texture, a method for generating arbitrarily large texture images from a text prompt. Our approach fine-tunes a diffusion model on a single texture, and learns to embed that statistical distribution in the output domain of the model. We seed this fine-tuning process with a sample texture patch, which can be optionally generated from a text-to-image model like DALL-E 2. At generation time, our fine-tuned diffusion model is used through a score aggregation strategy to generate output texture images of arbitrary resolution on a single GPU. We compare synthesized textures from our method to existing work in patch-based and deep learning texture synthesis methods. We also showcase two applications of our generated textures in 3D rendering and texture transfer.