I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image

TL;DR

I2L-MeshNet introduces an innovative line+pixel prediction approach that enhances 3D human pose and mesh estimation accuracy from single RGB images by preserving spatial relationships and modeling uncertainty.

Contribution

The paper presents a novel image-to-lixel prediction network that predicts per-lixel likelihoods on 1D heatmaps, improving over direct parameter regression methods.

Findings

Outperforms previous 3D human pose estimation methods

Preserves spatial relationships in the input image

Models prediction uncertainty effectively

Abstract

Most of the previous image-based 3D human pose and mesh estimation methods estimate parameters of the human mesh model from an input image. However, directly regressing the parameters from the input image is a highly non-linear mapping because it breaks the spatial relationship between pixels in the input image. In addition, it cannot model the prediction uncertainty, which can make training harder. To resolve the above issues, we propose I2L-MeshNet, an image-to-lixel (line+pixel) prediction network. The proposed I2L-MeshNet predicts the per-lixel likelihood on 1D heatmaps for each mesh vertex coordinate instead of directly regressing the parameters. Our lixel-based 1D heatmap preserves the spatial relationship in the input image and models the prediction uncertainty. We demonstrate the benefit of the image-to-lixel prediction and show that the proposed I2L-MeshNet outperforms previous methods. The code is publicly available https://github.com/mks0601/I2L-MeshNet_RELEASE.