Towards Practical Human Motion Prediction with LiDAR Point Clouds

TL;DR

This paper introduces LiDAR-HMP, a novel approach for 3D human motion prediction directly from raw LiDAR point clouds, overcoming limitations of prior methods that rely on pose estimation or sparse keypoints.

Contribution

LiDAR-HMP is the first single-LiDAR-based method that directly predicts future human poses from raw point clouds, utilizing a novel structure-aware feature descriptor for improved accuracy.

Findings

Achieves state-of-the-art results on public benchmarks

Demonstrates robustness in real-world scenarios

Effectively models spatial-temporal human motion correlations

Abstract

Human motion prediction is crucial for human-centric multimedia understanding and interacting. Current methods typically rely on ground truth human poses as observed input, which is not practical for real-world scenarios where only raw visual sensor data is available. To implement these methods in practice, a pre-phrase of pose estimation is essential. However, such two-stage approaches often lead to performance degradation due to the accumulation of errors. Moreover, reducing raw visual data to sparse keypoint representations significantly diminishes the density of information, resulting in the loss of fine-grained features. In this paper, we propose \textit{LiDAR-HMP}, the first single-LiDAR-based 3D human motion prediction approach, which receives the raw LiDAR point cloud as input and forecasts future 3D human poses directly. Building upon our novel structure-aware body feature descriptor, LiDAR-HMP adaptively maps the observed motion manifold to future poses and effectively models the spatial-temporal correlations of human motions for further refinement of prediction results. Extensive experiments show that our method achieves state-of-the-art performance on two public benchmarks and demonstrates remarkable robustness and efficacy in real-world deployments.