A Comparative Study of 3D Person Detection: Sensor Modalities and Robustness in Diverse Indoor and Outdoor Environments

TL;DR

This study systematically compares camera-only, LiDAR-only, and fusion-based 3D person detection methods across diverse indoor and outdoor environments, highlighting the advantages and vulnerabilities of each approach.

Contribution

It provides a comprehensive evaluation of detection performance and robustness of different sensor modalities and fusion in varied scenes using the JRDB dataset.

Findings

Fusion-based detection outperforms single-modality models in challenging scenarios.

DAL fusion approach shows improved resilience but remains sensitive to sensor misalignments.

Camera-only BEVDepth performs poorly under occlusion, noise, and distance variations.

Abstract

Accurate 3D person detection is critical for safety in applications such as robotics, industrial monitoring, and surveillance. This work presents a systematic evaluation of 3D person detection using camera-only, LiDAR-only, and camera-LiDAR fusion. While most existing research focuses on autonomous driving, we explore detection performance and robustness in diverse indoor and outdoor scenes using the JRDB dataset. We compare three representative models - BEVDepth (camera), PointPillars (LiDAR), and DAL (camera-LiDAR fusion) - and analyze their behavior under varying occlusion and distance levels. Our results show that the fusion-based approach consistently outperforms single-modality models, particularly in challenging scenarios. We further investigate robustness against sensor corruptions and misalignments, revealing that while DAL offers improved resilience, it remains sensitive to sensor misalignment and certain LiDAR-based corruptions. In contrast, the camera-based BEVDepth model showed the lowest performance and was most affected by occlusion, distance, and noise. Our findings highlight the importance of utilizing sensor fusion for enhanced 3D person detection, while also underscoring the need for ongoing research to address the vulnerabilities inherent in these systems.