D2Styler: Advancing Arbitrary Style Transfer with Discrete Diffusion Methods

TL;DR

D2Styler introduces a novel discrete diffusion framework leveraging VQ-GANs and AdaIN features to improve arbitrary style transfer, overcoming mode-collapse issues and producing higher quality stylized images.

Contribution

The paper presents D2Styler, a new discrete diffusion-based approach that enhances style transfer quality and stability by integrating VQ-GANs and adaptive normalization.

Findings

Outperforms twelve existing style transfer methods on multiple metrics

Produces visually appealing and high-quality stylized images

Demonstrates stable training and avoids mode collapse

Abstract

In image processing, one of the most challenging tasks is to render an image's semantic meaning using a variety of artistic approaches. Existing techniques for arbitrary style transfer (AST) frequently experience mode-collapse, over-stylization, or under-stylization due to a disparity between the style and content images. We propose a novel framework called D$^2$Styler (Discrete Diffusion Styler) that leverages the discrete representational capability of VQ-GANs and the advantages of discrete diffusion, including stable training and avoidance of mode collapse. Our method uses Adaptive Instance Normalization (AdaIN) features as a context guide for the reverse diffusion process. This makes it easy to move features from the style image to the content image without bias. The proposed method substantially enhances the visual quality of style-transferred images, allowing the combination of content and style in a visually appealing manner. We take style images from the WikiArt dataset and content images from the COCO dataset. Experimental results demonstrate that D$^2$Styler produces high-quality style-transferred images and outperforms twelve existing methods on nearly all the metrics. The qualitative results and ablation studies provide further insights into the efficacy of our technique. The code is available at https://github.com/Onkarsus13/D2Styler.