Lotus: Creating Short Videos From Long Videos With Abstractive and Extractive Summarization

TL;DR

Lotus is a system that combines abstractive and extractive summarization techniques to help creators efficiently produce engaging short videos from long videos, balancing content preservation and flexibility.

Contribution

We introduce Lotus, a novel system that integrates abstractive and extractive methods for automated short video creation from long videos, enhancing creator control and efficiency.

Findings

Lotus produces short videos comparable to human-created ones in user studies.

The system effectively balances content fidelity with creative flexibility.

Participants preferred Lotus-assisted editing over traditional methods.

Abstract

Short-form videos are popular on platforms like TikTok and Instagram as they quickly capture viewers' attention. Many creators repurpose their long-form videos to produce short-form videos, but creators report that planning, extracting, and arranging clips from long-form videos is challenging. Currently, creators make extractive short-form videos composed of existing long-form video clips or abstractive short-form videos by adding newly recorded narration to visuals. While extractive videos maintain the original connection between audio and visuals, abstractive videos offer flexibility in selecting content to be included in a shorter time. We present Lotus, a system that combines both approaches to balance preserving the original content with flexibility over the content. Lotus first creates an abstractive short-form video by generating both a short-form script and its corresponding speech, then matching long-form video clips to the generated narration. Creators can then add extractive clips with an automated method or Lotus's editing interface. Lotus's interface can be used to further refine the short-form video. We compare short-form videos generated by Lotus with those using an extractive baseline method. In our user study, we compare creating short-form videos using Lotus to participants' existing practice.