Interactive Drawing Guidance for Anime Illustrations with Diffusion Model

TL;DR

This paper introduces an interactive system that provides real-time guidance for creating anime illustrations, combining diffusion models and sketch refinement techniques to assist users, especially beginners, in producing high-quality artwork efficiently.

Contribution

It presents a novel real-time interactive drawing guidance system for anime art, integrating diffusion models, sketch transformation, and optimization for improved user assistance.

Findings

System significantly improves drawing accuracy and efficiency.

User study shows high usability and positive feedback.

Real-time guidance aligns well with user creative intent.

Abstract

Creating high-quality anime illustrations presents notable challenges, particularly for beginners, due to the intricate styles and fine details inherent in anime art. We present an interactive drawing guidance system specifically designed for anime illustrations to address this issue. It offers real-time guidance to help users refine their work and streamline the creative process. Our system is built upon the StreamDiffusion pipeline to deliver real-time drawing assistance. We fine-tune Stable Diffusion with LoRA to synthesize anime style RGB images from user-provided hand-drawn sketches and prompts. Leveraging the Informative Drawings model, we transform these RGB images into rough sketches, which are further refined into structured guidance sketches using a custom-designed optimizer. The proposed system offers precise, real-time guidance aligned with the creative intent of the user, significantly enhancing both the efficiency and accuracy of the drawing process. To assess the effectiveness of our approach, we conducted a user study, gathering empirical feedback on both system performance and interface usability.