LiDAR-HMR: 3D Human Mesh Recovery from LiDAR

TL;DR

This paper introduces LiDAR-HMR, a novel method for reconstructing 3D human body meshes from sparse LiDAR point clouds, addressing challenges of sparsity and noise with a cascaded graph transformer and a sparse-to-dense reconstruction scheme.

Contribution

It is the first to estimate 3D human meshes from LiDAR point clouds using a cascaded graph transformer and a sparse-to-dense reconstruction approach.

Findings

Effective reconstruction demonstrated on three datasets.

Outperforms existing methods in handling sparse and noisy point clouds.

Provides publicly available code for reproducibility.

Abstract

In recent years, point cloud perception tasks have been garnering increasing attention. This paper presents the first attempt to estimate 3D human body mesh from sparse LiDAR point clouds. We found that the major challenge in estimating human pose and mesh from point clouds lies in the sparsity, noise, and incompletion of LiDAR point clouds. Facing these challenges, we propose an effective sparse-to-dense reconstruction scheme to reconstruct 3D human mesh. This involves estimating a sparse representation of a human (3D human pose) and gradually reconstructing the body mesh. To better leverage the 3D structural information of point clouds, we employ a cascaded graph transformer (graphormer) to introduce point cloud features during sparse-to-dense reconstruction. Experimental results on three publicly available databases demonstrate the effectiveness of the proposed approach. Code: https://github.com/soullessrobot/LiDAR-HMR/