Finding Action Tubes

TL;DR

This paper introduces a method for detecting actions in videos by combining shape, kinematic cues, appearance, and motion features, resulting in efficient and accurate action tube predictions.

Contribution

It presents a novel approach that integrates motion saliency and deep learning features to improve action detection in videos.

Findings

Outperforms existing action detection techniques

Reduces computational load by selecting motion salient regions

Produces temporally consistent action tubes

Abstract

We address the problem of action detection in videos. Driven by the latest progress in object detection from 2D images, we build action models using rich feature hierarchies derived from shape and kinematic cues. We incorporate appearance and motion in two ways. First, starting from image region proposals we select those that are motion salient and thus are more likely to contain the action. This leads to a significant reduction in the number of regions being processed and allows for faster computations. Second, we extract spatio-temporal feature representations to build strong classifiers using Convolutional Neural Networks. We link our predictions to produce detections consistent in time, which we call action tubes. We show that our approach outperforms other techniques in the task of action detection.