3D Pose from Detections

TL;DR

This paper introduces a closed-form method to estimate 3D object positions and orientations from 2D detections across multiple views, utilizing ellipsoid fitting and dual-space algebraic solutions, with robustness enhancements.

Contribution

It presents a novel algebraic approach for 3D pose estimation from 2D detections, including a robust ellipse fitting algorithm to handle detection errors.

Findings

Effective in multi-view scenarios with minimal views

Robust to inaccuracies in 2D detections

Validated on synthetic and real datasets

Abstract

We present a novel method to infer, in closed-form, a general 3D spatial occupancy and orientation of a collection of rigid objects given 2D image detections from a sequence of images. In particular, starting from 2D ellipses fitted to bounding boxes, this novel multi-view problem can be reformulated as the estimation of a quadric (ellipsoid) in 3D. We show that an efficient solution exists in the dual-space using a minimum of three views while a solution with two views is possible through the use of regularization. However, this algebraic solution can be negatively affected in the presence of gross inaccuracies in the bounding boxes estimation. To this end, we also propose a robust ellipse fitting algorithm able to improve performance in the presence of errors in the detected objects. Results on synthetic tests and on different real datasets, involving real challenging scenarios, demonstrate the applicability and potential of our method.