MRStyle: A Unified Framework for Color Style Transfer with Multi-Modality Reference

TL;DR

MRStyle is a unified framework for color style transfer that effectively uses both image and text references, achieving high-quality, artifact-free results with efficient processing and style consistency.

Contribution

The paper introduces IRStyle for a unified style feature space and TRStyle for text-guided transfer, enabling multi-modality style transfer with improved quality and efficiency.

Findings

Outperforms state-of-the-art in image style transfer

Achieves artifact-free high-resolution results

Effective text-guided color style transfer

Abstract

In this paper, we introduce MRStyle, a comprehensive framework that enables color style transfer using multi-modality reference, including image and text. To achieve a unified style feature space for both modalities, we first develop a neural network called IRStyle, which generates stylized 3D lookup tables for image reference. This is accomplished by integrating an interaction dual-mapping network with a combined supervised learning pipeline, resulting in three key benefits: elimination of visual artifacts, efficient handling of high-resolution images with low memory usage, and maintenance of style consistency even in situations with significant color style variations. For text reference, we align the text feature of stable diffusion priors with the style feature of our IRStyle to perform text-guided color style transfer (TRStyle). Our TRStyle method is highly efficient in both training and inference, producing notable open-set text-guided transfer results. Extensive experiments in both image and text settings demonstrate that our proposed method outperforms the state-of-the-art in both qualitative and quantitative evaluations.