Deep Iterative 2D/3D Registration

TL;DR

This paper introduces a novel deep learning framework for 2D/3D registration that achieves high accuracy and robustness without additional refinement, suitable for clinical use with an average runtime of 8 seconds.

Contribution

It presents an end-to-end deep learning approach for iterative 2D/3D registration using Point-to-Plane Correspondences and residual refinement, eliminating the need for separate refinement steps.

Findings

Achieves 0.60 mm mean re-projection error

97% success ratio in registration

Runtime of approximately 8 seconds

Abstract

Deep Learning-based 2D/3D registration methods are highly robust but often lack the necessary registration accuracy for clinical application. A refinement step using the classical optimization-based 2D/3D registration method applied in combination with Deep Learning-based techniques can provide the required accuracy. However, it also increases the runtime. In this work, we propose a novel Deep Learning driven 2D/3D registration framework that can be used end-to-end for iterative registration tasks without relying on any further refinement step. We accomplish this by learning the update step of the 2D/3D registration framework using Point-to-Plane Correspondences. The update step is learned using iterative residual refinement-based optical flow estimation, in combination with the Point-to-Plane correspondence solver embedded as a known operator. Our proposed method achieves an average runtime of around 8s, a mean re-projection distance error of 0.60 $\pm$ 0.40 mm with a success ratio of 97 percent and a capture range of 60 mm. The combination of high registration accuracy, high robustness, and fast runtime makes our solution ideal for clinical applications.