DiffSketching: Sketch Control Image Synthesis with Diffusion Models
Qiang Wang, Di Kong, Fengyin Lin, Yonggang Qi

TL;DR
This paper introduces DiffSketching, a diffusion model-based approach for sketch-to-image synthesis that accurately translates abstract sketches into diverse, high-quality images without large datasets, outperforming GANs.
Contribution
The paper proposes a novel diffusion model framework for sketch-to-image synthesis that does not require large-scale datasets and improves accuracy and diversity over existing methods.
Findings
Outperforms GAN-based methods in quality and human evaluation
Maintains diversity and imagination in generated images
Effective in image editing and interpolation applications
Abstract
Creative sketch is a universal way of visual expression, but translating images from an abstract sketch is very challenging. Traditionally, creating a deep learning model for sketch-to-image synthesis needs to overcome the distorted input sketch without visual details, and requires to collect large-scale sketch-image datasets. We first study this task by using diffusion models. Our model matches sketches through the cross domain constraints, and uses a classifier to guide the image synthesis more accurately. Extensive experiments confirmed that our method can not only be faithful to user's input sketches, but also maintain the diversity and imagination of synthetic image results. Our model can beat GAN-based method in terms of generation quality and human evaluation, and does not rely on massive sketch-image datasets. Additionally, we present applications of our method in image editing…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Computer Graphics and Visualization Techniques · Advanced Vision and Imaging
MethodsDiffusion
