StableMaterials: Enhancing Diversity in Material Generation via Semi-Supervised Learning

TL;DR

StableMaterials is a semi-supervised learning approach that combines diffusion models and adversarial training to generate diverse, high-quality, and tileable PBR materials with minimal annotated data.

Contribution

It introduces a novel semi-supervised framework integrating diffusion models, adversarial training, and tileability techniques for diverse material generation.

Findings

Outperforms state-of-the-art methods in diversity and quality.

Enables fast, high-resolution material generation with minimal diffusion steps.

Produces tileable textures free of visual artifacts.

Abstract

We introduce StableMaterials, a novel approach for generating photorealistic physical-based rendering (PBR) materials that integrate semi-supervised learning with Latent Diffusion Models (LDMs). Our method employs adversarial training to distill knowledge from existing large-scale image generation models, minimizing the reliance on annotated data and enhancing the diversity in generation. This distillation approach aligns the distribution of the generated materials with that of image textures from an SDXL model, enabling the generation of novel materials that are not present in the initial training dataset. Furthermore, we employ a diffusion-based refiner model to improve the visual quality of the samples and achieve high-resolution generation. Finally, we distill a latent consistency model for fast generation in just four steps and propose a new tileability technique that removes visual artifacts typically associated with fewer diffusion steps. We detail the architecture and training process of StableMaterials, the integration of semi-supervised training within existing LDM frameworks and show the advantages of our approach. Comparative evaluations with state-of-the-art methods show the effectiveness of StableMaterials, highlighting its potential applications in computer graphics and beyond. StableMaterials is publicly available at https://gvecchio.com/stablematerials.