Zero-Shot Contrastive Loss for Text-Guided Diffusion Image Style Transfer

TL;DR

This paper introduces a zero-shot contrastive loss technique for diffusion models that enables effective text-guided image style transfer without additional training or neural networks, improving content preservation and versatility.

Contribution

The authors propose a novel zero-shot contrastive loss method for diffusion models that enhances style transfer and image translation without fine-tuning or extra networks.

Findings

Outperforms existing style transfer methods in content preservation.

Works effectively for image-to-image translation and manipulation.

Requires no additional training or fine-tuning.

Abstract

Diffusion models have shown great promise in text-guided image style transfer, but there is a trade-off between style transformation and content preservation due to their stochastic nature. Existing methods require computationally expensive fine-tuning of diffusion models or additional neural network. To address this, here we propose a zero-shot contrastive loss for diffusion models that doesn't require additional fine-tuning or auxiliary networks. By leveraging patch-wise contrastive loss between generated samples and original image embeddings in the pre-trained diffusion model, our method can generate images with the same semantic content as the source image in a zero-shot manner. Our approach outperforms existing methods while preserving content and requiring no additional training, not only for image style transfer but also for image-to-image translation and manipulation. Our experimental results validate the effectiveness of our proposed method.