Creating Multimedia Summaries Using Tweets and Videos
Anietie Andy, Siyi Liu, Daphne Ippolito, Reno Kriz, Chris, Callison-Burch, Derry Wijaya

TL;DR
This paper presents a method to generate multimedia summaries of televised events by combining social media commentary and video frames, focusing on spikes in mentions of event participants.
Contribution
It introduces a novel approach that automatically identifies key scenes based on social media activity and selects relevant tweets and video frames during those periods.
Findings
Effective scene identification based on mention spikes
Automatic selection of relevant tweets and video frames
Improved multimedia summarization of live events
Abstract
While popular televised events such as presidential debates or TV shows are airing, people provide commentary on them in real-time. In this paper, we propose a simple yet effective approach to combine social media commentary and videos to create a multimedia summary of televised events. Our approach identifies scenes from these events based on spikes of mentions of people involved in the event and automatically selects tweets and frames from the videos that occur during the time period of the spike that talk about and show the people being discussed.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsVideo Analysis and Summarization · Advanced Text Analysis Techniques · Multimodal Machine Learning Applications
