Texture Synthesis Using Shallow Convolutional Networks with Random Filters

TL;DR

This paper shows that shallow CNNs with random filters can effectively model and synthesize natural textures, rivaling more complex deep models in quality despite their simplicity.

Contribution

It demonstrates that random shallow CNNs are sufficient for high-quality texture synthesis, challenging the necessity of deep, trained networks for this task.

Findings

Random shallow CNNs classify texture patches effectively.

Synthesized textures capture large-scale spatial correlations.

Shallow networks can rival state-of-the-art texture models.

Abstract

Here we demonstrate that the feature space of random shallow convolutional neural networks (CNNs) can serve as a surprisingly good model of natural textures. Patches from the same texture are consistently classified as being more similar then patches from different textures. Samples synthesized from the model capture spatial correlations on scales much larger then the receptive field size, and sometimes even rival or surpass the perceptual quality of state of the art texture models (but show less variability). The current state of the art in parametric texture synthesis relies on the multi-layer feature space of deep CNNs that were trained on natural images. Our finding suggests that such optimized multi-layer feature spaces are not imperative for texture modeling. Instead, much simpler shallow and convolutional networks can serve as the basis for novel texture synthesis algorithms.