View Invariant Human Body Detection and Pose Estimation from Multiple Depth Sensors

TL;DR

This paper introduces Point R-CNN, an end-to-end multi-person 3D pose estimation network that effectively fuses multiple point cloud sources for indoor monitoring, outperforming existing methods especially in challenging scenarios.

Contribution

The paper presents a novel multi-sensor 3D pose estimation network that fuses point clouds at input level, improving robustness and accuracy over complex indoor scenes.

Findings

Outperforms state-of-the-art models in multi-sensor 3D pose estimation

Effective handling of sensor failures and cluttered scenes

Demonstrates robustness in real-world indoor scenarios

Abstract

Point cloud based methods have produced promising results in areas such as 3D object detection in autonomous driving. However, most of the recent point cloud work focuses on single depth sensor data, whereas less work has been done on indoor monitoring applications, such as operation room monitoring in hospitals or indoor surveillance. In these scenarios multiple cameras are often used to tackle occlusion problems. We propose an end-to-end multi-person 3D pose estimation network, Point R-CNN, using multiple point cloud sources. We conduct extensive experiments to simulate challenging real world cases, such as individual camera failures, various target appearances, and complex cluttered scenes with the CMU panoptic dataset and the MVOR operation room dataset. Unlike most of the previous methods that attempt to use multiple sensor information by building complex fusion models, which often lead to poor generalization, we take advantage of the efficiency of concatenating point clouds to fuse the information at the input level. In the meantime, we show our end-to-end network greatly outperforms cascaded state-of-the-art models.