Texture Synthesis Through Convolutional Neural Networks and Spectrum Constraints

TL;DR

This paper introduces a novel texture synthesis method combining CNN-based statistical features with Fourier spectrum constraints, improving large-scale coherence and fine details without extra computational cost.

Contribution

It integrates spectrum constraints into CNN-based texture synthesis, enhancing large-scale structure depiction while maintaining local detail, outperforming existing CNN-only methods.

Findings

Improved large-scale structure coherence in synthesized textures.

Enhanced fine-scale detail preservation.

Outperforms recent CNN-based texture synthesis methods.

Abstract

This paper presents a significant improvement for the synthesis of texture images using convolutional neural networks (CNNs), making use of constraints on the Fourier spectrum of the results. More precisely, the texture synthesis is regarded as a constrained optimization problem, with constraints conditioning both the Fourier spectrum and statistical features learned by CNNs. In contrast with existing methods, the presented method inherits from previous CNN approaches the ability to depict local structures and fine scale details, and at the same time yields coherent large scale structures, even in the case of quasi-periodic images. This is done at no extra computational cost. Synthesis experiments on various images show a clear improvement compared to a recent state-of-the art method relying on CNN constraints only.