Characterizing Multimedia Information Environment through Multi-modal Clustering of YouTube Videos

TL;DR

This paper introduces a multi-modal clustering framework for analyzing YouTube videos, revealing content themes, patterns, and repurposing techniques across audio, video, and text modalities, with a case study on South China Sea videos.

Contribution

It presents a novel multi-modal clustering approach for multimedia content characterization, integrating audio, video, and text analysis to uncover content themes and repurposing patterns.

Findings

Identified geopolitical and security themes in text modality.

Detected distinct patterns in news, educational, and interview videos.

Uncovered content repurposing and amplification techniques.

Abstract

This study aims to investigate the comprehensive characterization of information content in multimedia (videos), particularly on YouTube. The research presents a multi-method framework for characterizing multimedia content by clustering signals from various modalities, such as audio, video, and text. With a focus on South China Sea videos as a case study, this approach aims to enhance our understanding of online content, especially on YouTube. The dataset includes 160 videos, and our findings offer insights into content themes and patterns within different modalities of a video based on clusters. Text modality analysis revealed topical themes related to geopolitical countries, strategies, and global security, while video and audio modality analysis identified distinct patterns of signals related to diverse sets of videos, including news analysis/reporting, educational content, and interviews. Furthermore, our findings uncover instances of content repurposing within video clusters, which were identified using the barcode technique and audio similarity assessments. These findings indicate potential content amplification techniques. In conclusion, this study uniquely enhances our current understanding of multimedia content information based on modality clustering techniques.