Texture Synthesis Using Convolutional Neural Networks

TL;DR

This paper presents a neural network-based model for natural texture synthesis that leverages feature correlations across multiple layers, producing high-quality textures and offering insights into deep CNN representations.

Contribution

Introduces a novel texture synthesis method using CNN feature correlations, bridging computer vision and neuroscience insights.

Findings

Textures generated with high perceptual quality

Model captures statistical properties of natural images

Provides a new tool for neuroscience research

Abstract

Here we introduce a new model of natural textures based on the feature spaces of convolutional neural networks optimised for object recognition. Samples from the model are of high perceptual quality demonstrating the generative power of neural networks trained in a purely discriminative fashion. Within the model, textures are represented by the correlations between feature maps in several layers of the network. We show that across layers the texture representations increasingly capture the statistical properties of natural images while making object information more and more explicit. The model provides a new tool to generate stimuli for neuroscience and might offer insights into the deep representations learned by convolutional neural networks.