HPERL: 3D Human Pose Estimation from RGB and LiDAR

TL;DR

This paper introduces HPERL, an end-to-end method combining RGB and LiDAR data to improve absolute 3D human pose estimation accuracy, leveraging weak supervision from 2D annotations.

Contribution

The paper presents a novel architecture that integrates LiDAR with RGB for precise 3D human pose estimation and employs weak supervision to reduce annotation requirements.

Findings

Achieves unprecedented accuracy in 3D pose estimation using RGB and LiDAR.

Demonstrates effectiveness of weakly-supervised learning with 2D annotations.

Outperforms existing RGB-only and RGB-D methods in various scenarios.

Abstract

In-the-wild human pose estimation has a huge potential for various fields, ranging from animation and action recognition to intention recognition and prediction for autonomous driving. The current state-of-the-art is focused only on RGB and RGB-D approaches for predicting the 3D human pose. However, not using precise LiDAR depth information limits the performance and leads to very inaccurate absolute pose estimation. With LiDAR sensors becoming more affordable and common on robots and autonomous vehicle setups, we propose an end-to-end architecture using RGB and LiDAR to predict the absolute 3D human pose with unprecedented precision. Additionally, we introduce a weakly-supervised approach to generate 3D predictions using 2D pose annotations from PedX [1]. This allows for many new opportunities in the field of 3D human pose estimation.