Constructing Human Motion Manifold with Sequential Networks

TL;DR

This paper introduces a recurrent neural network approach to build a comprehensive human motion manifold that captures diverse motions and enables natural motion synthesis through sampling and algebraic operations.

Contribution

It proposes new regularization, dual decoders, and a forward kinematics layer to enhance the quality and versatility of the human motion manifold.

Findings

The motion manifold can generate diverse natural human motions.

The method enables effective interpolation and analogy in motion space.

The approach improves motion reconstruction accuracy.

Abstract

This paper presents a novel recurrent neural network-based method to construct a latent motion manifold that can represent a wide range of human motions in a long sequence. We introduce several new components to increase the spatial and temporal coverage in motion space while retaining the details of motion capture data. These include new regularization terms for the motion manifold, combination of two complementary decoders for predicting joint rotations and joint velocities, and the addition of the forward kinematics layer to consider both joint rotation and position errors. In addition, we propose a set of loss terms that improve the overall quality of the motion manifold from various aspects, such as the capability of reconstructing not only the motion but also the latent manifold vector, and the naturalness of the motion through adversarial loss. These components contribute to creating compact and versatile motion manifold that allows for creating new motions by performing random sampling and algebraic operations, such as interpolation and analogy, in the latent motion manifold.