Transfer Learning for Pose Estimation of Illustrated Characters

TL;DR

This paper develops a transfer learning approach to improve pose estimation for illustrated characters, creating new datasets and applying the model to pose-guided illustration retrieval.

Contribution

It introduces a transfer learning framework for illustrated character pose estimation, expanding datasets, and demonstrating its application in illustration retrieval.

Findings

Achieved state-of-the-art performance in illustrated pose estimation.

Expanded datasets for pose, classification, and segmentation tasks.

Enabled pose-guided illustration retrieval.

Abstract

Human pose information is a critical component in many downstream image processing tasks, such as activity recognition and motion tracking. Likewise, a pose estimator for the illustrated character domain would provide a valuable prior for assistive content creation tasks, such as reference pose retrieval and automatic character animation. But while modern data-driven techniques have substantially improved pose estimation performance on natural images, little work has been done for illustrations. In our work, we bridge this domain gap by efficiently transfer-learning from both domain-specific and task-specific source models. Additionally, we upgrade and expand an existing illustrated pose estimation dataset, and introduce two new datasets for classification and segmentation subtasks. We then apply the resultant state-of-the-art character pose estimator to solve the novel task of pose-guided illustration retrieval. All data, models, and code will be made publicly available.