Identity-Aware Semi-Supervised Learning for Comic Character Re-Identification

TL;DR

This paper presents a semi-supervised, identity-aware framework for comic character re-identification that combines metric learning with contrastive self-supervision, effectively handling limited annotations and complex appearance variations.

Contribution

It introduces a novel unified network architecture for face and body features, enabling robust character embeddings for improved re-identification in comics.

Findings

Outperforms face-only or body-only re-identification methods.

Effective on newly curated large-scale datasets.

Produces identity-aligned embeddings without face or body constraints.

Abstract

Character re-identification, recognizing characters consistently across different panels in comics, presents significant challenges due to limited annotated data and complex variations in character appearances. To tackle this issue, we introduce a robust semi-supervised framework that combines metric learning with a novel 'Identity-Aware' self-supervision method by contrastive learning of face and body pairs of characters. Our approach involves processing both facial and bodily features within a unified network architecture, facilitating the extraction of identity-aligned character embeddings that capture individual identities while preserving the effectiveness of face and body features. This integrated character representation enhances feature extraction and improves character re-identification compared to re-identification by face or body independently, offering a parameter-efficient solution. By extensively validating our method using in-series and inter-series evaluation metrics, we demonstrate its effectiveness in consistently re-identifying comic characters. Compared to existing methods, our approach not only addresses the challenge of character re-identification but also serves as a foundation for downstream tasks since it can produce character embeddings without restrictions of face and body availability, enriching the comprehension of comic books. In our experiments, we leverage two newly curated datasets: the 'Comic Character Instances Dataset', comprising over a million character instances and the 'Comic Sequence Identity Dataset', containing annotations of identities within more than 3000 sets of four consecutive comic panels that we collected.