Learning Linear Transformations for Fast Arbitrary Style Transfer

TL;DR

This paper introduces a fast, learnable linear transformation approach for arbitrary style transfer that efficiently combines multi-level styles while maintaining content structure, outperforming existing methods in quality and speed.

Contribution

The authors derive a theoretical form of the transformation matrix and propose a feed-forward network to learn it, enabling flexible, efficient style transfer across various tasks.

Findings

Achieves high-quality style transfer with reduced computational cost

Supports multi-level style blending while preserving content integrity

Demonstrates superior performance on artistic, video, photo-realistic, and domain adaptation tasks

Abstract

Given a random pair of images, an arbitrary style transfer method extracts the feel from the reference image to synthesize an output based on the look of the other content image. Recent arbitrary style transfer methods transfer second order statistics from reference image onto content image via a multiplication between content image features and a transformation matrix, which is computed from features with a pre-determined algorithm. These algorithms either require computationally expensive operations, or fail to model the feature covariance and produce artifacts in synthesized images. Generalized from these methods, in this work, we derive the form of transformation matrix theoretically and present an arbitrary style transfer approach that learns the transformation matrix with a feed-forward network. Our algorithm is highly efficient yet allows a flexible combination of multi-level styles while preserving content affinity during style transfer process. We demonstrate the effectiveness of our approach on four tasks: artistic style transfer, video and photo-realistic style transfer as well as domain adaptation, including comparisons with the state-of-the-art methods.