"Previously on ..." From Recaps to Story Summarization
Aditya Kumar Singh, Dhruv Srivastava, Makarand Tapaswi

TL;DR
This paper presents PlotSnap, a new dataset and TaleSumm, a hierarchical model for multimodal story summarization that extracts multiple plot points from long TV episodes using recaps and dialogue analysis.
Contribution
It introduces PlotSnap dataset and a hierarchical model TaleSumm for multimodal story summarization, enabling extraction of multiple plot points from long videos.
Findings
TaleSumm effectively predicts importance scores for shots and dialogues.
The approach generalizes well across different TV series.
TaleSumm performs competitively on traditional video summarization benchmarks.
Abstract
We introduce multimodal story summarization by leveraging TV episode recaps - short video sequences interweaving key story moments from previous episodes to bring viewers up to speed. We propose PlotSnap, a dataset featuring two crime thriller TV shows with rich recaps and long episodes of 40 minutes. Story summarization labels are unlocked by matching recap shots to corresponding sub-stories in the episode. We propose a hierarchical model TaleSumm that processes entire episodes by creating compact shot and dialog representations, and predicts importance scores for each video shot and dialog utterance by enabling interactions between local story groups. Unlike traditional summarization, our method extracts multiple plot points from long videos. We present a thorough evaluation on story summarization, including promising cross-series generalization. TaleSumm also shows good results on…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDigital Humanities and Scholarship · Natural Language Processing Techniques
