Cartoondiff: Training-free Cartoon Image Generation with Diffusion Transformer Models

TL;DR

CartoonDiff is a training-free diffusion transformer-based method for image cartoonization that normalizes high-frequency signals during denoising, eliminating the need for retraining or reference images.

Contribution

We introduce CartoonDiff, a novel training-free approach that decomposes diffusion processes for effective cartoonization without additional data or complex tuning.

Findings

Achieves high-quality cartoonization without retraining

Operates without reference images or complex parameters

Demonstrates strong results across diverse images

Abstract

Image cartoonization has attracted significant interest in the field of image generation. However, most of the existing image cartoonization techniques require re-training models using images of cartoon style. In this paper, we present CartoonDiff, a novel training-free sampling approach which generates image cartoonization using diffusion transformer models. Specifically, we decompose the reverse process of diffusion models into the semantic generation phase and the detail generation phase. Furthermore, we implement the image cartoonization process by normalizing high-frequency signal of the noisy image in specific denoising steps. CartoonDiff doesn't require any additional reference images, complex model designs, or the tedious adjustment of multiple parameters. Extensive experimental results show the powerful ability of our CartoonDiff. The project page is available at: https://cartoondiff.github.io/