A causal convolutional neural network for multi-subject motion modeling and generation

TL;DR

This paper introduces a causal convolutional neural network inspired by WaveNet for multi-subject motion modeling and synthesis, capable of capturing individual motion characteristics and adapting to new skeletons with fine-tuning.

Contribution

The proposed neural network effectively models multi-subject motion characteristics and enables personalized motion synthesis for unseen skeletons through fine-tuning.

Findings

Successfully captures intrinsic motion features across subjects

Enables high-quality personalized motion synthesis for new skeletons

Demonstrates versatility in various motion modeling tasks

Abstract

Inspired by the success of WaveNet in multi-subject speech synthesis, we propose a novel neural network based on causal convolutions for multi-subject motion modeling and generation. The network can capture the intrinsic characteristics of the motion of different subjects, such as the influence of skeleton scale variation on motion style. Moreover, after fine-tuning the network using a small motion dataset for a novel skeleton that is not included in the training dataset, it is able to synthesize high-quality motions with a personalized style for the novel skeleton. The experimental results demonstrate that our network can model the intrinsic characteristics of motions well and can be applied to various motion modeling and synthesis tasks.