Co-occurrence Based Texture Synthesis

TL;DR

This paper introduces a controllable, interpretable texture synthesis method using a co-occurrence conditioned GAN, enabling large, smooth, and user-adjustable textures with a novel differentiable loss.

Contribution

It presents a new co-occurrence conditioned GAN framework with a differentiable loss for stable, interpretable, and controllable texture synthesis.

Findings

Stable training with the proposed loss.

Smooth texture morphing demonstrated.

Interactive texture editing enabled.

Abstract

As image generation techniques mature, there is a growing interest in explainable representations that are easy to understand and intuitive to manipulate. In this work, we turn to co-occurrence statistics, which have long been used for texture analysis, to learn a controllable texture synthesis model. We propose a fully convolutional generative adversarial network, conditioned locally on co-occurrence statistics, to generate arbitrarily large images while having local, interpretable control over the texture appearance. To encourage fidelity to the input condition, we introduce a novel differentiable co-occurrence loss that is integrated seamlessly into our framework in an end-to-end fashion. We demonstrate that our solution offers a stable, intuitive and interpretable latent representation for texture synthesis, which can be used to generate a smooth texture morph between different textures. We further show an interactive texture tool that allows a user to adjust local characteristics of the synthesized texture image using the co-occurrence values directly.