Systematic Comparison of Projection Methods for Monocular 3D Human Pose Estimation on Fisheye Images

TL;DR

This paper systematically evaluates various projection methods for monocular 3D human pose estimation in fisheye images, highlighting the importance of choosing the right model for different FOV scenarios and introducing a new dataset.

Contribution

It provides a comprehensive comparison of projection models for fisheye-based pose estimation and proposes a heuristic for model selection, along with a new dataset for evaluation.

Findings

Double sphere model improves accuracy in wide FOV scenarios

Pinhole projection is inadequate for close-up images

Heuristic improves prediction quality by selecting appropriate projection

Abstract

Fisheye cameras offer robots the ability to capture human movements across a wider field of view (FOV) than standard pinhole cameras, making them particularly useful for applications in human-robot interaction and automotive contexts. However, accurately detecting human poses in fisheye images is challenging due to the curved distortions inherent to fisheye optics. While various methods for undistorting fisheye images have been proposed, their effectiveness and limitations for poses that cover a wide FOV has not been systematically evaluated in the context of absolute human pose estimation from monocular fisheye images. To address this gap, we evaluate the impact of pinhole, equidistant and double sphere camera models, as well as cylindrical projection methods, on 3D human pose estimation accuracy. We find that in close-up scenarios, pinhole projection is inadequate, and the optimal projection method varies with the FOV covered by the human pose. The usage of advanced fisheye models like the double sphere model significantly enhances 3D human pose estimation accuracy. We propose a heuristic for selecting the appropriate projection model based on the detection bounding box to enhance prediction quality. Additionally, we introduce and evaluate on our novel dataset FISHnCHIPS, which features 3D human skeleton annotations in fisheye images, including images from unconventional angles, such as extreme close-ups, ground-mounted cameras, and wide-FOV poses, available at: https://www.vision.rwth-aachen.de/fishnchips