A survey of exemplar-based texture synthesis

TL;DR

This survey reviews exemplar-based texture synthesis methods, including statistical, patch-based, hybrid, and neural approaches, highlighting their strengths and limitations in modeling multi-scale natural textures.

Contribution

It provides a comprehensive overview of existing texture synthesis techniques and discusses the challenges in modeling complex, multi-scale natural textures.

Findings

Neural methods produce impressive results on various textures.

Most real textures are multi-scale, posing challenges for current methods.

State-of-the-art techniques struggle with large natural images.

Abstract

Exemplar-based texture synthesis is the process of generating, from an input sample, new texture images of arbitrary size and which are perceptually equivalent to the sample. The two main approaches are statistics-based methods and patch re-arrangement methods. In the first class, a texture is characterized by a statistical signature; then, a random sampling conditioned to this signature produces genuinely different texture images. The second class boils down to a clever "copy-paste" procedure, which stitches together large regions of the sample. Hybrid methods try to combine ideas from both approaches to avoid their hurdles. The recent approaches using convolutional neural networks fit to this classification, some being statistical and others performing patch re-arrangement in the feature space. They produce impressive synthesis on various kinds of textures. Nevertheless, we found that most real textures are organized at multiple scales, with global structures revealed at coarse scales and highly varying details at finer ones. Thus, when confronted with large natural images of textures the results of state-of-the-art methods degrade rapidly, and the problem of modeling them remains wide open.