Dynamic Character Graph via Online Face Clustering for Movie Analysis

TL;DR

This paper introduces an unsupervised method to build a dynamic character interaction graph from movies, capturing temporal character interactions for improved narrative analysis and character retrieval.

Contribution

The paper presents a novel online face clustering algorithm combined with dynamic graph construction, enabling real-time analysis of character interactions in movies.

Findings

Achieves superior performance in narrative segmentation

Enhances major character retrieval accuracy

Effective on movies with over 5000 face tracks

Abstract

An effective approach to automated movie content analysis involves building a network (graph) of its characters. Existing work usually builds a static character graph to summarize the content using metadata, scripts or manual annotations. We propose an unsupervised approach to building a dynamic character graph that captures the temporal evolution of character interaction. We refer to this as the character interaction graph(CIG). Our approach has two components:(i) an online face clustering algorithm that discovers the characters in the video stream as they appear, and (ii) simultaneous creation of a CIG using the temporal dynamics of the resulting clusters. We demonstrate the usefulness of the CIG for two movie analysis tasks: narrative structure (acts) segmentation, and major character retrieval. Our evaluation on full-length movies containing more than 5000 face tracks shows that the proposed approach achieves superior performance for both the tasks.