Utilizing Task-Generic Motion Prior to Recover Full-Body Motion from Very Sparse Signals

TL;DR

This paper introduces a neural motion prior-based method to improve full-body motion reconstruction from sparse VR tracking signals, enhancing accuracy especially in lower body pose recovery.

Contribution

It presents a novel approach that predicts the overall motion latent representation from limited signals, integrating it with sensor data for better full-body pose estimation.

Findings

Improved accuracy in full-body motion reconstruction.

Enhanced robustness in lower body motion recovery.

Effective integration of motion prior with sparse sensor data.

Abstract

The most popular type of devices used to track a user's posture in a virtual reality experience consists of a head-mounted display and two controllers held in both hands. However, due to the limited number of tracking sensors (three in total), faithfully recovering the user in full-body is challenging, limiting the potential for interactions among simulated user avatars within the virtual world. Therefore, recent studies have attempted to reconstruct full-body poses using neural networks that utilize previously learned human poses or accept a series of past poses over a short period. In this paper, we propose a method that utilizes information from a neural motion prior to improve the accuracy of reconstructed user's motions. Our approach aims to reconstruct user's full-body poses by predicting the latent representation of the user's overall motion from limited input signals and integrating this information with tracking sensor inputs. This is based on the premise that the ultimate goal of pose reconstruction is to reconstruct the motion, which is a series of poses. Our results show that this integration enables more accurate reconstruction of the user's full-body motion, particularly enhancing the robustness of lower body motion reconstruction from impoverished signals. Web: https://https://mjsh34.github.io/mp-sspe/