DynaWeightPnP: Toward global real-time 3D-2D solver in PnP without correspondences

TL;DR

This paper introduces DynaWeightPnP, a real-time, correspondence-free 3D-2D pose estimation method using RKHS and dynamic weighting, achieving high speed and accuracy suitable for medical and robotic applications.

Contribution

The study presents DynaWeightPnP, a novel approach combining RKHS and dynamic weighting to solve correspondence-free PnP efficiently and accurately in real-time.

Findings

Achieves 60 Hz registration rate without post-refinement.

Demonstrates competitive accuracy with existing methods.

Effective in vascular centerline registration for EIGIs.

Abstract

This paper addresses a special Perspective-n-Point (PnP) problem: estimating the optimal pose to align 3D and 2D shapes in real-time without correspondences, termed as correspondence-free PnP. While several studies have focused on 3D and 2D shape registration, achieving both real-time and accurate performance remains challenging. This study specifically targets the 3D-2D geometric shape registration tasks, applying the recently developed Reproducing Kernel Hilbert Space (RKHS) to address the "big-to-small" issue. An iterative reweighted least squares method is employed to solve the RKHS-based formulation efficiently. Moreover, our work identifies a unique and interesting observability issue in correspondence-free PnP: the numerical ambiguity between rotation and translation. To address this, we proposed DynaWeightPnP, introducing a dynamic weighting sub-problem and an alternative searching algorithm designed to enhance pose estimation and alignment accuracy. Experiments were conducted on a typical case, that is, a 3D-2D vascular centerline registration task within Endovascular Image-Guided Interventions (EIGIs). Results demonstrated that the proposed algorithm achieves registration processing rates of 60 Hz (without post-refinement) and 31 Hz (with post-refinement) on modern single-core CPUs, with competitive accuracy comparable to existing methods. These results underscore the suitability of DynaWeightPnP for future robot navigation tasks like EIGIs.