SketchAssist: A Practical Assistant for Semantic Edits and Precise Local Redrawing

TL;DR

SketchAssist is a versatile interactive tool that combines instruction-guided global edits with line-guided local redrawing, enhancing digital sketch editing by preserving style and structure while enabling semantic modifications.

Contribution

The paper introduces a scalable data generation pipeline and a unified editing framework with a task-guided mixture-of-experts, enabling precise, style-preserving sketch editing with minimal changes to existing models.

Findings

Achieves state-of-the-art results in sketch editing tasks.

Demonstrates superior style and structure preservation.

Provides a practical, controllable sketch editing assistant.

Abstract

Sketch editing is central to digital illustration, yet existing image editing systems struggle to preserve the sparse, style-sensitive structure of line art while supporting both high-level semantic changes and precise local redrawing. We present SketchAssist, an interactive sketch drawing assistant that accelerates creation by unifying instruction-guided global edits with line-guided region redrawing, while keeping unrelated regions and overall composition intact. To enable this assistant at scale, we introduce a controllable data generation pipeline that (i) constructs attribute-addition sequences from attribute-free base sketches, (ii) forms multi-step edit chains via cross-sequence sampling, and (iii) expands stylistic coverage with a style-preserving attribute-removal model applied to diverse sketches. Building on this data, SketchAssist employs a unified sketch editing framework with minimal changes to DiT-based editors. We repurpose the RGB channels to encode the inputs, enabling seamless switching between instruction-guided edits and line-guided redrawing within a single input interface. To further specialize behavior across modes, we integrate a task-guided mixture-of-experts into LoRA layers, routing by text and visual cues to improve semantic controllability, structural fidelity, and style preservation. Extensive experiments show state-of-the-art results on both tasks, with superior instruction adherence and style/structure preservation compared to recent baselines. Together, our dataset and SketchAssist provide a practical, controllable assistant for sketch creation and revision.