SketcherX: AI-Driven Interactive Robotic drawing with Diffusion model and Vectorization Techniques

TL;DR

SketcherX is an innovative robotic system that combines AI models and vectorization techniques to create personalized, stylized portraits through interactive human-robot collaboration, achieving high-quality results rapidly.

Contribution

The paper introduces a new robotic drawing system utilizing diffusion models and a custom LoRA for style adaptation, advancing interactive robotic art.

Findings

Produces high-quality portraits within two minutes

Demonstrates effective style adaptation with LoRA

Enhances stroke quality through pair-wise fine-tuning

Abstract

We introduce SketcherX, a novel robotic system for personalized portrait drawing through interactive human-robot engagement. Unlike traditional robotic art systems that rely on analog printing techniques, SketcherX captures and processes facial images to produce vectorized drawings in a distinctive, human-like artistic style. The system comprises two 6-axis robotic arms : a face robot, which is equipped with a head-mounted camera and Large Language Model (LLM) for real-time interaction, and a drawing robot, utilizing a fine-tuned Stable Diffusion model, ControlNet, and Vision-Language models for dynamic, stylized drawing. Our contributions include the development of a custom Vector Low Rank Adaptation model (LoRA), enabling seamless adaptation to various artistic styles, and integrating a pair-wise fine-tuning approach to enhance stroke quality and stylistic accuracy. Experimental results demonstrate the system's ability to produce high-quality, personalized portraits within two minutes, highlighting its potential as a new paradigm in robotic creativity. This work advances the field of robotic art by positioning robots as active participants in the creative process, paving the way for future explorations in interactive, human-robot artistic collaboration.