Structured Pattern Expansion with Diffusion Models

TL;DR

This paper introduces a novel diffusion model-based method for controllably expanding structured, stationary patterns from partial sketches into larger, tileable designs, enhancing pattern synthesis quality and user control.

Contribution

It adapts diffusion models for pattern synthesis by fine-tuning with LoRA, employing noise rolling for tileability, and using patch-based generation for large-scale assets.

Findings

Outperforms existing models in pattern diversity and consistency

Enables direct user control over pattern expansion

Produces tileable, high-quality large-scale patterns

Abstract

Recent advances in diffusion models have significantly improved the synthesis of materials, textures, and 3D shapes. By conditioning these models via text or images, users can guide the generation, reducing the time required to create digital assets. In this paper, we address the synthesis of structured, stationary patterns, where diffusion models are generally less reliable and, more importantly, less controllable. Our approach leverages the generative capabilities of diffusion models specifically adapted for the pattern domain. It enables users to exercise direct control over the synthesis by expanding a partially hand-drawn pattern into a larger design while preserving the structure and details of the input. To enhance pattern quality, we fine-tune an image-pretrained diffusion model on structured patterns using Low-Rank Adaptation (LoRA), apply a noise rolling technique to ensure tileability, and utilize a patch-based approach to facilitate the generation of large-scale assets. We demonstrate the effectiveness of our method through a comprehensive set of experiments, showing that it outperforms existing models in generating diverse, consistent patterns that respond directly to user input.