Discovery and recognition of motion primitives in human activities

TL;DR

This paper introduces an unsupervised framework for discovering and recognizing human motion primitives from 3D pose data, enabling automatic categorization and recognition of complex human activities in videos.

Contribution

The novel framework combines motion flux optimization, normalization, and hierarchical Bayesian clustering to discover and recognize motion primitives without prior labels.

Findings

Built a publicly available dataset of human motion primitives.

Achieved unsupervised discovery and classification of motion primitives.

Enabled probabilistic recognition of primitives in new video data.

Abstract

We present a novel framework for the automatic discovery and recognition of motion primitives in videos of human activities. Given the 3D pose of a human in a video, human motion primitives are discovered by optimizing the `motion flux', a quantity which captures the motion variation of a group of skeletal joints. A normalization of the primitives is proposed in order to make them invariant with respect to a subject anatomical variations and data sampling rate. The discovered primitives are unknown and unlabeled and are unsupervisedly collected into classes via a hierarchical non-parametric Bayes mixture model. Once classes are determined and labeled they are further analyzed for establishing models for recognizing discovered primitives. Each primitive model is defined by a set of learned parameters. Given new video data and given the estimated pose of the subject appearing on the video, the motion is segmented into primitives, which are recognized with a probability given according to the parameters of the learned models. Using our framework we build a publicly available dataset of human motion primitives, using sequences taken from well-known motion capture datasets. We expect that our framework, by providing an objective way for discovering and categorizing human motion, will be a useful tool in numerous research fields including video analysis, human inspired motion generation, learning by demonstration, intuitive human-robot interaction, and human behavior analysis.