STALP: Style Transfer with Auxiliary Limited Pairing

TL;DR

This paper introduces STALP, a real-time style transfer method that uses a single stylized pair and auxiliary unpaired images to produce consistent, semantically meaningful stylizations across various media types.

Contribution

STALP is the first approach to leverage a single stylized pair along with unpaired images for stable, real-time style transfer across diverse content.

Findings

Achieves temporally stable stylization without explicit temporal constraints.

Effectively transfers style to videos, panoramas, faces, and 3D models.

Preserves source style characteristics better than concurrent methods.

Abstract

We present an approach to example-based stylization of images that uses a single pair of a source image and its stylized counterpart. We demonstrate how to train an image translation network that can perform real-time semantically meaningful style transfer to a set of target images with similar content as the source image. A key added value of our approach is that it considers also consistency of target images during training. Although those have no stylized counterparts, we constrain the translation to keep the statistics of neural responses compatible with those extracted from the stylized source. In contrast to concurrent techniques that use a similar input, our approach better preserves important visual characteristics of the source style and can deliver temporally stable results without the need to explicitly handle temporal consistency. We demonstrate its practical utility on various applications including video stylization, style transfer to panoramas, faces, and 3D models.