Unsupervised Manga Character Re-identification via Face-body and Spatial-temporal Associated Clustering

TL;DR

This paper introduces an unsupervised method for manga character re-identification that leverages face-body relationships and spatial-temporal information to improve clustering accuracy amidst artistic variability.

Contribution

The paper presents a novel Face-body and Spatial-temporal Associated Clustering (FSAC) method that addresses artistic challenges in manga re-identification without labeled data.

Findings

Outperforms existing methods on a large manga dataset

Effectively handles artistic exaggeration and deformation

Achieves high clustering accuracy in unsupervised setting

Abstract

In the past few years, there has been a dramatic growth in e-manga (electronic Japanese-style comics). Faced with the booming demand for manga research and the large amount of unlabeled manga data, we raised a new task, called unsupervised manga character re-identification. However, the artistic expression and stylistic limitations of manga pose many challenges to the re-identification problem. Inspired by the idea that some content-related features may help clustering, we propose a Face-body and Spatial-temporal Associated Clustering method (FSAC). In the face-body combination module, a face-body graph is constructed to solve problems such as exaggeration and deformation in artistic creation by using the integrity of the image. In the spatial-temporal relationship correction module, we analyze the appearance features of characters and design a temporal-spatial-related triplet loss to fine-tune the clustering. Extensive experiments on a manga book dataset with 109 volumes validate the superiority of our method in unsupervised manga character re-identification.