Latent-Info and Low-Dimensional Learning for Human Mesh Recovery and Parallel Optimization

TL;DR

This paper introduces a two-stage network for human mesh recovery that leverages latent information and low-dimensional learning to improve accuracy and reduce computational costs, especially in complex scenes.

Contribution

The proposed method extracts hybrid latent frequency features and employs a low-dimensional, parallel optimization approach for mesh pose interaction, advancing accuracy and efficiency.

Findings

Outperforms state-of-the-art methods on large datasets.

Effectively reduces computational costs.

Improves local detail reconstruction in complex scenes.

Abstract

Existing 3D human mesh recovery methods often fail to fully exploit the latent information (e.g., human motion, shape alignment), leading to issues with limb misalignment and insufficient local details in the reconstructed human mesh (especially in complex scenes). Furthermore, the performance improvement gained by modelling mesh vertices and pose node interactions using attention mechanisms comes at a high computational cost. To address these issues, we propose a two-stage network for human mesh recovery based on latent information and low dimensional learning. Specifically, the first stage of the network fully excavates global (e.g., the overall shape alignment) and local (e.g., textures, detail) information from the low and high-frequency components of image features and aggregates this information into a hybrid latent frequency domain feature. This strategy effectively extracts latent information. Subsequently, utilizing extracted hybrid latent frequency domain features collaborates to enhance 2D poses to 3D learning. In the second stage, with the assistance of hybrid latent features, we model the interaction learning between the rough 3D human mesh template and the 3D pose, optimizing the pose and shape of the human mesh. Unlike existing mesh pose interaction methods, we design a low-dimensional mesh pose interaction method through dimensionality reduction and parallel optimization that significantly reduces computational costs without sacrificing reconstruction accuracy. Extensive experimental results on large publicly available datasets indicate superiority compared to the most state-of-the-art.