VisioBlend: Sketch and Stroke-Guided Denoising Diffusion Probabilistic Model for Realistic Image Generation

TL;DR

VisioBlend is a diffusion-based framework that allows users to generate realistic images from sketches and strokes with adjustable fidelity, overcoming data limitations and enhancing image synthesis flexibility.

Contribution

It introduces a unified diffusion model supporting 3D control over sketch and stroke-guided image generation, improving realism and diversity in content creation.

Findings

Achieves state-of-the-art realism in sketch-to-image synthesis

Enables adjustable faithfulness to input sketches and strokes

Synthesizes new data to improve robustness and diversity

Abstract

Generating images from hand-drawings is a crucial and fundamental task in content creation. The translation is challenging due to the infinite possibilities and the diverse expectations of users. However, traditional methods are often limited by the availability of training data. Therefore, VisioBlend, a unified framework supporting three-dimensional control over image synthesis from sketches and strokes based on diffusion models, is proposed. It enables users to decide the level of faithfulness to the input strokes and sketches. VisioBlend achieves state-of-the-art performance in terms of realism and flexibility, enabling various applications in image synthesis from sketches and strokes. It solves the problem of data availability by synthesizing new data points from hand-drawn sketches and strokes, enriching the dataset and enabling more robust and diverse image synthesis. This work showcases the power of diffusion models in image creation, offering a user-friendly and versatile approach for turning artistic visions into reality.