SPAST: Arbitrary Style Transfer with Style Priors via Pre-trained Large-scale Model

TL;DR

SPAST is a new style transfer framework that combines style priors from large-scale models with a local-global stylization module to produce high-quality images efficiently.

Contribution

It introduces a novel style prior loss and a local-global window size stylization module to improve quality and reduce inference time in arbitrary style transfer.

Findings

Outperforms state-of-the-art methods in image quality

Reduces inference time significantly

Effectively preserves content structure

Abstract

Given an arbitrary content and style image, arbitrary style transfer aims to render a new stylized image which preserves the content image's structure and possesses the style image's style. Existing arbitrary style transfer methods are based on either small models or pre-trained large-scale models. The small model-based methods fail to generate high-quality stylized images, bringing artifacts and disharmonious patterns. The pre-trained large-scale model-based methods can generate high-quality stylized images but struggle to preserve the content structure and cost long inference time. To this end, we propose a new framework, called SPAST, to generate high-quality stylized images with less inference time. Specifically, we design a novel Local-global Window Size Stylization Module (LGWSSM)tofuse style features into content features. Besides, we introduce a novel style prior loss, which can dig out the style priors from a pre-trained large-scale model into the SPAST and motivate the SPAST to generate high-quality stylized images with short inference time.We conduct abundant experiments to verify that our proposed method can generate high-quality stylized images and less inference time compared with the SOTA arbitrary style transfer methods.