A Certifably Correct Algorithm for Generalized Robot-World and Hand-Eye Calibration

TL;DR

This paper introduces a fast, globally optimal algorithm for generalized robot-world and hand-eye calibration, supporting multiple sensors and monocular cameras, with theoretical guarantees and open-source code.

Contribution

A novel certifiably correct algorithm for generalized RWHEC that handles multiple sensors, monocular cameras, and provides theoretical guarantees of optimality.

Findings

Superior performance over existing methods in simulations and real experiments.

Derived new identifiability criteria and global optimality guarantees.

Proposed a novel constraint qualification for nonlinear programs with redundant constraints.

Abstract

Automatic extrinsic sensor calibration is a fundamental problem for multi-sensor platforms. Reliable and general-purpose solutions should be computationally efficient, require few assumptions about the structure of the sensing environment, and demand little effort from human operators. In this work, we introduce a fast and certifiably globally optimal algorithm for solving a generalized formulation of the robot-world and hand-eye calibration (RWHEC) problem. The formulation of RWHEC presented is "generalized" in that it supports the simultaneous estimation of multiple sensor and target poses, and permits the use of monocular cameras that, alone, are unable to measure the scale of their environments. In addition to demonstrating our method's superior performance over existing solutions through extensive simulated and real experiments, we derive novel identifiability criteria and establish a priori guarantees of global optimality for problem instances with bounded measurement errors. As part of our analysis, we propose a new constraint qualification for nonlinear programs with redundant constraints; this constraint qualification is of independent interest for establishing the exactness of SDP relaxations of QCQPs that have been tightened through the addition of redundant constraints. Finally, we provide a free and open-source implementation of our algorithms and experiments.