Compositional Neural Textures

TL;DR

This paper presents an unsupervised neural model that represents textures as compositions of Gaussian textons, enabling easy editing, synthesis, and transfer of textures in images.

Contribution

It introduces a novel compositional neural texture model using Gaussian textons for unsupervised texture representation and editing.

Findings

Allows intuitive texture editing by modifying Gaussian components

Enables efficient texture synthesis through a feed-forward generator

Supports diverse applications like texture transfer and animation

Abstract

Texture plays a vital role in enhancing visual richness in both real photographs and computer-generated imagery. However, the process of editing textures often involves laborious and repetitive manual adjustments of textons, which are the recurring local patterns that characterize textures. This work introduces a fully unsupervised approach for representing textures using a compositional neural model that captures individual textons. We represent each texton as a 2D Gaussian function whose spatial support approximates its shape, and an associated feature that encodes its detailed appearance. By modeling a texture as a discrete composition of Gaussian textons, the representation offers both expressiveness and ease of editing. Textures can be edited by modifying the compositional Gaussians within the latent space, and new textures can be efficiently synthesized by feeding the modified Gaussians through a generator network in a feed-forward manner. This approach enables a wide range of applications, including transferring appearance from an image texture to another image, diversifying textures,texture interpolation, revealing/modifying texture variations, edit propagation, texture animation, and direct texton manipulation. The proposed approach contributes to advancing texture analysis, modeling, and editing techniques, and opens up new possibilities for creating visually appealing images with controllable textures.