SketchColour: Channel Concat Guided DiT-based Sketch-to-Colour Pipeline for 2D Animation

TL;DR

SketchColour is a novel sketch-to-colour pipeline for 2D animation that leverages a diffusion transformer backbone with channel concatenation and LoRA finetuning, achieving high-quality, temporally coherent results with reduced resource usage.

Contribution

It introduces the first diffusion transformer-based sketch-to-colour pipeline for 2D animation, improving efficiency and quality over previous methods.

Findings

Outperforms state-of-the-art video colourization methods on SAKUGA dataset.

Uses half the training data of competing models.

Produces temporally coherent animations with minimal artifacts.

Abstract

The production of high-quality 2D animation is highly labor-intensive process, as animators are currently required to draw and color a large number of frames by hand. We present SketchColour, the first sketch-to-colour pipeline for 2D animation built on a diffusion transformer (DiT) backbone. By replacing the conventional U-Net denoiser with a DiT-style architecture and injecting sketch information via lightweight channel-concatenation adapters accompanied with LoRA finetuning, our method natively integrates conditioning without the parameter and memory bloat of a duplicated ControlNet, greatly reducing parameter count and GPU memory usage. Evaluated on the SAKUGA dataset, SketchColour outperforms previous state-of-the-art video colourization methods across all metrics, despite using only half the training data of competing models. Our approach produces temporally coherent animations with minimal artifacts such as colour bleeding or object deformation. Our code is available at: https://bconstantine.github.io/SketchColour .