LiveHPS: LiDAR-based Scene-level Human Pose and Shape Estimation in Free Environment

TL;DR

LiveHPS is a novel LiDAR-based method for accurate scene-level human pose and shape estimation that leverages temporal-spatial information and a distillation mechanism, enabling robust performance in diverse real-world environments.

Contribution

We introduce LiveHPS, a single-LiDAR approach with a distillation mechanism and temporal-spatial modeling, and present the large-scale FreeMotion dataset for human pose estimation.

Findings

Achieves state-of-the-art performance on multiple datasets.

Robustly handles occlusion and noise in LiDAR data.

Effective in diverse real-world scenarios.

Abstract

For human-centric large-scale scenes, fine-grained modeling for 3D human global pose and shape is significant for scene understanding and can benefit many real-world applications. In this paper, we present LiveHPS, a novel single-LiDAR-based approach for scene-level human pose and shape estimation without any limitation of light conditions and wearable devices. In particular, we design a distillation mechanism to mitigate the distribution-varying effect of LiDAR point clouds and exploit the temporal-spatial geometric and dynamic information existing in consecutive frames to solve the occlusion and noise disturbance. LiveHPS, with its efficient configuration and high-quality output, is well-suited for real-world applications. Moreover, we propose a huge human motion dataset, named FreeMotion, which is collected in various scenarios with diverse human poses, shapes and translations. It consists of multi-modal and multi-view acquisition data from calibrated and synchronized LiDARs, cameras, and IMUs. Extensive experiments on our new dataset and other public datasets demonstrate the SOTA performance and robustness of our approach. We will release our code and dataset soon.