Hyperbolic Space Learning Method Leveraging Temporal Motion Priors for Human Mesh Recovery

TL;DR

This paper introduces a hyperbolic space learning approach that leverages temporal motion priors to improve the accuracy and stability of 3D human mesh recovery from videos, capturing hierarchical structures more effectively.

Contribution

It proposes a novel hyperbolic space optimization strategy combined with temporal motion priors for better 3D human mesh reconstruction from videos.

Findings

Outperforms state-of-the-art methods on large datasets.

Effectively captures hierarchical human body structures.

Provides stable and accurate mesh optimization in hyperbolic space.

Abstract

3D human meshes show a natural hierarchical structure (like torso-limbs-fingers). But existing video-based 3D human mesh recovery methods usually learn mesh features in Euclidean space. It's hard to catch this hierarchical structure accurately. So wrong human meshes are reconstructed. To solve this problem, we propose a hyperbolic space learning method leveraging temporal motion prior for recovering 3D human meshes from videos. First, we design a temporal motion prior extraction module. This module extracts the temporal motion features from the input 3D pose sequences and image feature sequences respectively. Then it combines them into the temporal motion prior. In this way, it can strengthen the ability to express features in the temporal motion dimension. Since data representation in non-Euclidean space has been proved to effectively capture hierarchical relationships in real-world datasets (especially in hyperbolic space), we further design a hyperbolic space optimization learning strategy. This strategy uses the temporal motion prior information to assist learning, and uses 3D pose and pose motion information respectively in the hyperbolic space to optimize and learn the mesh features. Then, we combine the optimized results to get an accurate and smooth human mesh. Besides, to make the optimization learning process of human meshes in hyperbolic space stable and effective, we propose a hyperbolic mesh optimization loss. Extensive experimental results on large publicly available datasets indicate superiority in comparison with most state-of-the-art.