# Multiview 2D/3D Rigid Registration via a Point-Of-Interest Network for Tracking and Triangulation ($\text{POINT}^2$)

## TL;DR

This paper introduces $ ext{POINT}^2$, a novel learning-based framework for efficient and robust multiview 2D/3D rigid registration in medical interventions, outperforming existing methods in accuracy and speed.

## Contribution

$ ext{POINT}^2$ jointly learns 2D point correspondences and 3D pose estimation in a single forward pass, enhancing robustness and efficiency over prior approaches.

## Key findings

- Outperforms existing learning-based methods in accuracy and robustness.
- Requires only one forward pass for registration, reducing computation time.
- Improves the speed and robustness of optimization-based methods when used as an initial estimator.

## Abstract

We propose to tackle the problem of multiview 2D/3D rigid registration for intervention via a Point-Of-Interest Network for Tracking and Triangulation ($\text{POINT}^2$). $\text{POINT}^2$ learns to establish 2D point-to-point correspondences between the pre- and intra-intervention images by tracking a set of random POIs. The 3D pose of the pre-intervention volume is then estimated through a triangulation layer. In $\text{POINT}^2$, the unified framework of the POI tracker and the triangulation layer enables learning informative 2D features and estimating 3D pose jointly. In contrast to existing approaches, $\text{POINT}^2$ only requires a single forward-pass to achieve a reliable 2D/3D registration. As the POI tracker is shift-invariant, $\text{POINT}^2$ is more robust to the initial pose of the 3D pre-intervention image. Extensive experiments on a large-scale clinical cone-beam CT (CBCT) dataset show that the proposed $\text{POINT}^2$ method outperforms the existing learning-based method in terms of accuracy, robustness and running time. Furthermore, when used as an initial pose estimator, our method also improves the robustness and speed of the state-of-the-art optimization-based approaches by ten folds.