Articulated motion discovery using pairs of trajectories

TL;DR

This paper introduces an unsupervised motion pattern discovery method for articulated objects in videos, using trajectory pairs to identify and segment natural behaviors without supervision.

Contribution

It presents a novel trajectory pair-based descriptor that outperforms existing features in behavior recognition and segmentation in unconstrained animal videos.

Findings

Descriptor outperforms HOG and DTFs on datasets

Automatically segments videos into behavior-specific intervals

Effective on wild animal videos like tigers and dogs

Abstract

We propose an unsupervised approach for discovering characteristic motion patterns in videos of highly articulated objects performing natural, unscripted behaviors, such as tigers in the wild. We discover consistent patterns in a bottom-up manner by analyzing the relative displacements of large numbers of ordered trajectory pairs through time, such that each trajectory is attached to a different moving part on the object. The pairs of trajectories descriptor relies entirely on motion and is more discriminative than state-of-the-art features that employ single trajectories. Our method generates temporal video intervals, each automatically trimmed to one instance of the discovered behavior, and clusters them by type (e.g., running, turning head, drinking water). We present experiments on two datasets: dogs from YouTube-Objects and a new dataset of National Geographic tiger videos. Results confirm that our proposed descriptor outperforms existing appearance- and trajectory-based descriptors (e.g., HOG and DTFs) on both datasets and enables us to segment unconstrained animal video into intervals containing single behaviors.