TruNet: Short Videos Generation from Long Videos via Story-Preserving Truncation

TL;DR

This paper introduces a novel problem of story-preserving long video truncation, presents a new dataset called TruNet, and proposes a neural framework combining Boundary Aware Network and FF-LSTM to generate coherent short videos.

Contribution

The paper defines a new task, creates the TruNet dataset, and develops a neural architecture that outperforms existing methods in story-preserving video truncation.

Findings

Proposed method achieves higher accuracy than baselines.

TruNet dataset contains 1470 videos with detailed annotations.

Framework maintains story coherence in truncated videos.

Abstract

In this work, we introduce a new problem, named as {\em story-preserving long video truncation}, that requires an algorithm to automatically truncate a long-duration video into multiple short and attractive sub-videos with each one containing an unbroken story. This differs from traditional video highlight detection or video summarization problems in that each sub-video is required to maintain a coherent and integral story, which is becoming particularly important for resource-production video sharing platforms such as Youtube, Facebook, TikTok, Kwai, etc. To address the problem, we collect and annotate a new large video truncation dataset, named as TruNet, which contains 1470 videos with on average 11 short stories per video. With the new dataset, we further develop and train a neural architecture for video truncation that consists of two components: a Boundary Aware Network (BAN) and a Fast-Forward Long Short-Term Memory (FF-LSTM). We first use the BAN to generate high quality temporal proposals by jointly considering frame-level attractiveness and boundaryness. We then apply the FF-LSTM, which tends to capture high-order dependencies among a sequence of frames, to decide whether a temporal proposal is a coherent and integral story. We show that our proposed framework outperforms existing approaches for the story-preserving long video truncation problem in both quantitative measures and user-study. The dataset is available for public academic research usage at https://ai.baidu.com/broad/download.