Recurrent Semi-supervised Classification and Constrained Adversarial Generation with Motion Capture Data

TL;DR

This paper introduces recurrent encoder multi-decoder neural networks for semi-supervised motion classification and generation, improving generalization with limited labeled data and incorporating physical constraints for realistic future motion prediction.

Contribution

It proposes a novel recurrent semi-supervised classification framework with multiple reconstruction modules and a constrained recurrent GAN for physically plausible motion prediction.

Findings

Multiple reconstruction modules improve classification with limited labels

Constraints stabilize training of recurrent adversarial networks

Models enable clustering and faster labeling of motion data

Abstract

We explore recurrent encoder multi-decoder neural network architectures for semi-supervised sequence classification and reconstruction. We find that the use of multiple reconstruction modules helps models generalize in a classification task when only a small amount of labeled data is available, which is often the case in practice. Such models provide useful high-level representations of motions allowing clustering, searching and faster labeling of new sequences. We also propose a new, realistic partitioning of a well-known, high quality motion-capture dataset for better evaluations. We further explore a novel formulation for future-predicting decoders based on conditional recurrent generative adversarial networks, for which we propose both soft and hard constraints for transition generation derived from desired physical properties of synthesized future movements and desired animation goals. We find that using such constraints allow to stabilize the training of recurrent adversarial architectures for animation generation.