Self Supervised Networks for Learning Latent Space Representations of Human Body Scans and Motions

TL;DR

This paper presents self-supervised neural networks for efficient 3D human body shape and motion analysis, enabling fast retrieval, interpolation, and generation of body meshes and poses with minimal computational resources.

Contribution

The paper introduces VariShaPE for robust latent shape and pose embedding and MoGeN for learning body motion geometry within the latent space, advancing 3D human body analysis methods.

Findings

Fast and robust latent space retrieval of body shapes and poses

Effective motion interpolation, extrapolation, and transfer

Limited computational cost for various 3D body operations

Abstract

This paper introduces self-supervised neural network models to tackle several fundamental problems in the field of 3D human body analysis and processing. First, we propose VariShaPE (Varifold Shape Parameter Estimator), a novel architecture for the retrieval of latent space representations of body shapes and poses. This network offers a fast and robust method to estimate the embedding of arbitrary unregistered meshes into the latent space. Second, we complement the estimation of latent codes with MoGeN (Motion Geometry Network) a framework that learns the geometry on the latent space itself. This is achieved by lifting the body pose parameter space into a higher dimensional Euclidean space in which body motion mini-sequences from a training set of 4D data can be approximated by simple linear interpolation. Using the SMPL latent space representation we illustrate how the combination of these network models, once trained, can be used to perform a variety of tasks with very limited computational cost. This includes operations such as motion interpolation, extrapolation and transfer as well as random shape and pose generation.