Texture Representation via Analysis and Synthesis with Generative Adversarial Networks

TL;DR

This paper explores data-driven texture modeling using GANs, demonstrating how StyleGAN3 can synthesize diverse textures and introducing novel analysis techniques for both synthesized and real textures.

Contribution

It introduces a new analysis-synthesis framework for textures using GAN inversion and refinement, advancing texture modeling beyond existing methods.

Findings

StyleGAN3 can generate diverse textures beyond training data

Proposed GAN inversion with a novel latent domain reconstruction consistency criterion

Iterative refinement with Gramian loss improves real texture analysis

Abstract

We investigate data-driven texture modeling via analysis and synthesis with generative adversarial networks. For network training and testing, we have compiled a diverse set of spatially homogeneous textures, ranging from stochastic to regular. We adopt StyleGAN3 for synthesis and demonstrate that it produces diverse textures beyond those represented in the training data. For texture analysis, we propose GAN inversion using a novel latent domain reconstruction consistency criterion for synthesized textures, and iterative refinement with Gramian loss for real textures. We propose perceptual procedures for evaluating network capabilities, exploring the global and local behavior of latent space trajectories, and comparing with existing texture analysis-synthesis techniques.