Lightweight texture transfer based on texture feature preset

TL;DR

This paper introduces a lightweight texture transfer method that uses preset universal texture feature maps, significantly reducing model size and computation while maintaining high-quality results by leveraging the repetitive nature of textures.

Contribution

The proposed Texture Feature Preset (TFP) method efficiently encodes repetitive texture features, enabling faster and smaller models for texture transfer without sacrificing visual quality.

Findings

Reduces model size by up to 3538 times

Speeds up inference by up to 5.6 times

Produces visually superior texture transfer results

Abstract

In the task of texture transfer, reference texture images typically exhibit highly repetitive texture features, and the texture transfer results from different content images under the same style also share remarkably similar texture patterns. Encoding such highly similar texture features often requires deep layers and a large number of channels, making it is also the main source of the entire model's parameter count and computational load, and inference time. We propose a lightweight texture transfer based on texture feature preset (TFP). TFP takes full advantage of the high repetitiveness of texture features by providing preset universal texture feature maps for a given style. These preset feature maps can be fused and decoded directly with shallow color transfer feature maps of any content to generate texture transfer results, thereby avoiding redundant texture information from being encoded repeatedly. The texture feature map we preset is encoded through noise input images with consistent distribution (standard normal distribution). This consistent input distribution can completely avoid the problem of texture transfer differentiation, and by randomly sampling different noise inputs, we can obtain different texture features and texture transfer results under the same reference style. Compared to state-of-the-art techniques, our TFP not only produces visually superior results but also reduces the model size by 3.2-3538 times and speeds up the process by 1.8-5.6 times.