Towards real-time 6D pose estimation of objects in single-view cone-beam X-ray

TL;DR

This paper presents a real-time 6D object pose estimation method for single-view cone-beam X-ray images, using a refined RGB-based model trained on real data, achieving high accuracy with minimal training data.

Contribution

The authors adapt an RGB-based pose estimation model for X-ray images, enabling real-time 6D pose estimation with minimal training data and accounting for variable X-ray acquisition parameters.

Findings

93% accuracy at 5cm/5° threshold

Average 3D rotation error of 2.2 degrees

Requires significantly less training data than state-of-the-art

Abstract

Deep learning-based pose estimation algorithms can successfully estimate the pose of objects in an image, especially in the field of color images. 6D Object pose estimation based on deep learning models for X-ray images often use custom architectures that employ extensive CAD models and simulated data for training purposes. Recent RGB-based methods opt to solve pose estimation problems using small datasets, making them more attractive for the X-ray domain where medical data is scarcely available. We refine an existing RGB-based model (SingleShotPose) to estimate the 6D pose of a marked cube from grayscale X-ray images by creating a generic solution trained on only real X-ray data and adjusted for X-ray acquisition geometry. The model regresses 2D control points and calculates the pose through 2D/3D correspondences using Perspective-n-Point(PnP), allowing a single trained model to be used across all supporting cone-beam-based X-ray geometries. Since modern X-ray systems continuously adjust acquisition parameters during a procedure, it is essential for such a pose estimation network to consider these parameters in order to be deployed successfully and find a real use case. With a 5-cm/5-degree accuracy of 93% and an average 3D rotation error of 2.2 degrees, the results of the proposed approach are comparable with state-of-the-art alternatives, while requiring significantly less real training examples and being applicable in real-time applications.