Deep Translation Prior: Test-time Training for Photorealistic Style Transfer

TL;DR

The paper introduces Deep Translation Prior, a test-time training framework for photorealistic style transfer that learns image-specific translation priors without pre-training, improving generalization and performance on unseen images.

Contribution

It proposes a novel test-time training approach with specialized network architectures and loss functions, eliminating the need for offline training in style transfer.

Findings

Outperforms state-of-the-art methods in style transfer quality.

Demonstrates superior generalization to unseen image pairs.

Does not require pre-training, enabling flexible application.

Abstract

Recent techniques to solve photorealistic style transfer within deep convolutional neural networks (CNNs) generally require intensive training from large-scale datasets, thus having limited applicability and poor generalization ability to unseen images or styles. To overcome this, we propose a novel framework, dubbed Deep Translation Prior (DTP), to accomplish photorealistic style transfer through test-time training on given input image pair with untrained networks, which learns an image pair-specific translation prior and thus yields better performance and generalization. Tailored for such test-time training for style transfer, we present novel network architectures, with two sub-modules of correspondence and generation modules, and loss functions consisting of contrastive content, style, and cycle consistency losses. Our framework does not require offline training phase for style transfer, which has been one of the main challenges in existing methods, but the networks are to be solely learned during test-time. Experimental results prove that our framework has a better generalization ability to unseen image pairs and even outperforms the state-of-the-art methods.