Pose-dependent weights and Domain Randomization for fully automatic X-ray to CT Registration

TL;DR

This paper introduces a fully automatic X-ray to CT registration method that uses neural networks with domain randomization for landmark detection and a pose initialization scheme, enabling accurate registration on real and simulated data.

Contribution

It proposes a novel automatic initialization approach combining neural network landmark detection with domain randomization and patient-specific adaptation for end-to-end registration.

Findings

Achieved mean registration errors around 4.1-4.2 mm.

Success rates of approximately 87-92% on real and simulated X-rays.

Effective landmark detection despite training solely on simulated data.

Abstract

Fully automatic X-ray to CT registration requires a solid initialization to provide an initial alignment within the capture range of existing intensity-based registrations. This work adresses that need by providing a novel automatic initialization, which enables end to end registration. First, a neural network is trained once to detect a set of anatomical landmarks on simulated X-rays. A domain randomization scheme is proposed to enable the network to overcome the challenge of being trained purely on simulated data and run inference on real Xrays. Then, for each patient CT, a patient-specific landmark extraction scheme is used. It is based on backprojecting and clustering the previously trained networks predictions on a set of simulated X-rays. Next, the network is retrained to detect the new landmarks. Finally the combination of network and 3D landmark locations is used to compute the initialization using a perspective-n-point algorithm. During the computation of the pose, a weighting scheme is introduced to incorporate the confidence of the network in detecting the landmarks. The algorithm is evaluated on the pelvis using both real and simulated x-rays. The mean (+-standard deviation) target registration error in millimetres is 4.1 +- 4.3 for simulated X-rays with a success rate of 92% and 4.2 +- 3.9 for real X-rays with a success rate of 86.8%, where a success is defined as a translation error of less than 30mm.