Creating Multimedia Summaries Using Tweets and Videos

TL;DR

This paper presents a method to generate multimedia summaries of televised events by combining social media commentary and video frames, focusing on spikes in mentions of event participants.

Contribution

It introduces a novel approach that automatically identifies key scenes based on social media activity and selects relevant tweets and video frames during those periods.

Findings

Effective scene identification based on mention spikes

Automatic selection of relevant tweets and video frames

Improved multimedia summarization of live events

Abstract

While popular televised events such as presidential debates or TV shows are airing, people provide commentary on them in real-time. In this paper, we propose a simple yet effective approach to combine social media commentary and videos to create a multimedia summary of televised events. Our approach identifies scenes from these events based on spikes of mentions of people involved in the event and automatically selects tweets and frames from the videos that occur during the time period of the spike that talk about and show the people being discussed.