# Deep hashing for global registration of preoperative CT and video images for laparoscopic liver surgery

## TL;DR

This paper introduces a deep hashing method to automatically align preoperative CT scans with laparoscopic video images during liver surgery, improving augmented reality accuracy.

## Contribution

The first use of deep hashing for CT-to-video registration in laparoscopic liver surgery, enabling automatic and robust initialisation.

## Key findings

- Phantom experiments showed acceptable registration errors when sufficient pre-operative solutions were considered.
- The method achieved clinically relevant alignment in seven out of eight patient cases.
- Deep hashing effectively replaces manual initialisation across multiple views.

## Abstract

Registration of computed tomography (CT) to laparoscopic video images is vital to enable augmented reality (AR), a technology that holds the promise of minimising the risk of complications during laparoscopic liver surgery. Although several solutions have been presented in the literature, they always rely on an accurate initialisation of the registration that is either obtained manually or automatically estimated on very specific views of the liver. These limitations pose a challenge to the clinical translation of AR.

We propose the use of a content-based image retrieval (CBIR) framework to obtain an automatic robust initialisation to the registration. Instead of directly registering video and CT, we render a dense set of possible views of the liver from CT and extract liver contour features. To reduce feature maps to lower dimension vectors, we use a deep hashing (DH) network that is trained in a triplet scheme. Registration is obtained by matching the intra-operative image hashing encoding to the closest encodings found in the pre-operative renderings.

We validate our method on synthetic and real data from a phantom and real patient data from eight surgeries. Phantom experiments show that registration errors acceptable for an initial registration are obtained if sufficient pre-operative solutions are considered. In seven out of eight patients, the method is able to obtain a clinically relevant alignment.

We present the first work to adapt DH to the CT to video registration problem. Our results indicate that this framework can effectively replace manual initialisations in multiple views, potentially increasing the translation of these techniques.

The online version contains supplementary material available at 10.1007/s11548-025-03418-w.