Finding the Right Moment: Human-Assisted Trailer Creation via Task Composition

TL;DR

This paper presents a human-assisted, graph-based method for identifying trailer moments in movies, combining narrative and sentiment analysis with an interactive tool that outperforms automatic methods and rivals expert manual selection.

Contribution

It introduces a novel graph model with contrastive training for trailer shot selection, enabling interpretable and efficient human-in-the-loop trailer creation.

Findings

Unsupervised graph traversal effectively identifies preferred trailer shots.

The interactive tool reduces trailer creation time to under 30 minutes.

Selected trailer shots outperform fully automatic methods and match expert selections.

Abstract

Movie trailers perform multiple functions: they introduce viewers to the story, convey the mood and artistic style of the film, and encourage audiences to see the movie. These diverse functions make trailer creation a challenging endeavor. In this work, we focus on finding trailer moments in a movie, i.e., shots that could be potentially included in a trailer. We decompose this task into two subtasks: narrative structure identification and sentiment prediction. We model movies as graphs, where nodes are shots and edges denote semantic relations between them. We learn these relations using joint contrastive training which distills rich textual information (e.g., characters, actions, situations) from screenplays. An unsupervised algorithm then traverses the graph and selects trailer moments from the movie that human judges prefer to ones selected by competitive supervised approaches. A main advantage of our algorithm is that it uses interpretable criteria, which allows us to deploy it in an interactive tool for trailer creation with a human in the loop. Our tool allows users to select trailer shots in under 30 minutes that are superior to fully automatic methods and comparable to (exclusive) manual selection by experts.