LinkTo-Anime: A 2D Animation Optical Flow Dataset from 3D Model Rendering

TL;DR

LinkTo-Anime is a novel high-quality dataset tailored for optical flow estimation in cel anime character motion, generated via 3D rendering, supporting research in anime video tasks.

Contribution

This paper introduces the first dataset specifically designed for anime character motion optical flow, with extensive annotations and a benchmark for method evaluation.

Findings

Optical flow estimation methods show limitations on anime datasets.

LinkTo-Anime enhances research for anime video generation and line drawing colorization.

Benchmark reveals gaps in current optical flow techniques for anime content.

Abstract

Existing optical flow datasets focus primarily on real-world simulation or synthetic human motion, but few are tailored to Celluloid(cel) anime character motion: a domain with unique visual and motion characteristics. To bridge this gap and facilitate research in optical flow estimation and downstream tasks such as anime video generation and line drawing colorization, we introduce LinkTo-Anime, the first high-quality dataset specifically designed for cel anime character motion generated with 3D model rendering. LinkTo-Anime provides rich annotations including forward and backward optical flow, occlusion masks, and Mixamo Skeleton. The dataset comprises 395 video sequences, totally 24,230 training frames, 720 validation frames, and 4,320 test frames. Furthermore, a comprehensive benchmark is constructed with various optical flow estimation methods to analyze the shortcomings and limitations across multiple datasets.