Beat-Event Detection in Action Movie Franchises

TL;DR

This paper introduces a new dataset and method for detecting and classifying semantically defined beat-events in action movies, improving accuracy by leveraging shot classification and temporal constraints.

Contribution

The paper presents a novel dataset of Hollywood action movies with annotated beat-events and a new approach that combines shot classification with temporal constraints for better beat-event detection.

Findings

Temporal constraints improve classification accuracy

New dataset with annotated beat-events in action movies

Evaluation protocol for beat-event localization

Abstract

While important advances were recently made towards temporally localizing and recognizing specific human actions or activities in videos, efficient detection and classification of long video chunks belonging to semantically defined categories such as "pursuit" or "romance" remains challenging.We introduce a new dataset, Action Movie Franchises, consisting of a collection of Hollywood action movie franchises. We define 11 non-exclusive semantic categories - called beat-categories - that are broad enough to cover most of the movie footage. The corresponding beat-events are annotated as groups of video shots, possibly overlapping.We propose an approach for localizing beat-events based on classifying shots into beat-categories and learning the temporal constraints between shots. We show that temporal constraints significantly improve the classification performance. We set up an evaluation protocol for beat-event localization as well as for shot classification, depending on whether movies from the same franchise are present or not in the training data.