Story Understanding in Video Advertisements

TL;DR

This paper investigates the dynamic narrative structure of video advertisements, focusing on climax detection and sentiment prediction, and introduces a new model that significantly outperforms existing methods.

Contribution

The study crowdsources climax annotations, develops methods to predict climax points, and creates a sentiment prediction model that leverages context features and LSTM dynamics, surpassing prior state-of-the-art.

Findings

Climax prediction accuracy improved by 25% over previous models.

Context features and LSTM dynamics are essential for performance.

Crowdsourced annotations enable better understanding of ad narratives.

Abstract

In order to resonate with the viewers, many video advertisements explore creative narrative techniques such as "Freytag's pyramid" where a story begins with exposition, followed by rising action, then climax, concluding with denouement. In the dramatic structure of ads in particular, climax depends on changes in sentiment. We dedicate our study to understand the dynamic structure of video ads automatically. To achieve this, we first crowdsource climax annotations on 1,149 videos from the Video Ads Dataset, which already provides sentiment annotations. We then use both unsupervised and supervised methods to predict the climax. Based on the predicted peak, the low-level visual and audio cues, and semantically meaningful context features, we build a sentiment prediction model that outperforms the current state-of-the-art model of sentiment prediction in video ads by 25%. In our ablation study, we show that using our context features, and modeling dynamics with an LSTM, are both crucial factors for improved performance.