Kalib: Easy Hand-Eye Calibration with Reference Point Tracking

TL;DR

Kalib is an easy-to-use hand-eye calibration method that uses visual foundation models and minimal prerequisites to accurately estimate camera-robot transformations without retraining or detailed models.

Contribution

We introduce Kalib, a novel calibration approach leveraging foundation models, requiring only the robot's kinematic chain and a reference point, simplifying setup and broadening applicability.

Findings

Achieves accurate calibration in simulated and real-world tests.

Reduces manual effort compared to baseline methods.

Demonstrates effectiveness across various robot configurations.

Abstract

Hand-eye calibration aims to estimate the transformation between a camera and a robot. Traditional methods rely on fiducial markers, which require considerable manual effort and precise setup. Recent advances in deep learning have introduced markerless techniques but come with more prerequisites, such as retraining networks for each robot, and accessing accurate mesh models for data generation. In this paper, we propose Kalib, an automatic and easy-to-setup hand-eye calibration method that leverages the generalizability of visual foundation models to overcome these challenges. It features only two basic prerequisites, the robot's kinematic chain and a predefined reference point on the robot. During calibration, the reference point is tracked in the camera space. Its corresponding 3D coordinates in the robot coordinate can be inferred by forward kinematics. Then, a PnP solver directly estimates the transformation between the camera and the robot without training new networks or accessing mesh models. Evaluations in simulated and real-world benchmarks show that Kalib achieves good accuracy with a lower manual workload compared with recent baseline methods. We also demonstrate its application in multiple real-world settings with various robot arms and grippers. Kalib's user-friendly design and minimal setup requirements make it a possible solution for continuous operation in unstructured environments.