Representation learning of vertex heatmaps for 3D human mesh reconstruction from multi-view images

TL;DR

This paper introduces a vertex heatmap autoencoder that learns a manifold of plausible human meshes, enhancing 3D human mesh reconstruction from multi-view images and achieving state-of-the-art results.

Contribution

The paper proposes a novel vertex heatmap autoencoder that leverages large-scale motion data to improve mesh reconstruction accuracy.

Findings

Outperforms previous methods on Human3.6M and LightStage datasets.

Achieves state-of-the-art performance in 3D human mesh reconstruction.

Utilizes latent code supervision for better mesh estimation.

Abstract

This study addresses the problem of 3D human mesh reconstruction from multi-view images. Recently, approaches that directly estimate the skinned multi-person linear model (SMPL)-based human mesh vertices based on volumetric heatmap representation from input images have shown good performance. We show that representation learning of vertex heatmaps using an autoencoder helps improve the performance of such approaches. Vertex heatmap autoencoder (VHA) learns the manifold of plausible human meshes in the form of latent codes using AMASS, which is a large-scale motion capture dataset. Body code predictor (BCP) utilizes the learned body prior from VHA for human mesh reconstruction from multi-view images through latent code-based supervision and transfer of pretrained weights. According to experiments on Human3.6M and LightStage datasets, the proposed method outperforms previous methods and achieves state-of-the-art human mesh reconstruction performance.