ProtoRes: Proto-Residual Network for Pose Authoring via Learned Inverse Kinematics

TL;DR

ProtoRes introduces a neural network architecture that effectively infers complete human poses from sparse inputs, enhancing AI-assisted animation with improved accuracy and efficiency, and includes a Unity interface and new datasets.

Contribution

The paper presents a novel Proto-Residual Network combining residuals and prototype encoding for pose completion, outperforming Transformer baselines in accuracy and speed.

Findings

Outperforms Transformer baseline in accuracy

Achieves higher computational efficiency

Provides publicly available datasets and code

Abstract

Our work focuses on the development of a learnable neural representation of human pose for advanced AI assisted animation tooling. Specifically, we tackle the problem of constructing a full static human pose based on sparse and variable user inputs (e.g. locations and/or orientations of a subset of body joints). To solve this problem, we propose a novel neural architecture that combines residual connections with prototype encoding of a partially specified pose to create a new complete pose from the learned latent space. We show that our architecture outperforms a baseline based on Transformer, both in terms of accuracy and computational efficiency. Additionally, we develop a user interface to integrate our neural model in Unity, a real-time 3D development platform. Furthermore, we introduce two new datasets representing the static human pose modeling problem, based on high-quality human motion capture data, which will be released publicly along with model code.