Learning-Based Keypoint Registration for Fetoscopic Mosaicking

TL;DR

This paper introduces a learning-based keypoint registration framework for fetoscopic mosaicking, improving field-of-view expansion during TTTS surgeries by providing surgeons with better context awareness.

Contribution

It presents a novel learning-based approach with keypoint proposal and filtering strategies, outperforming existing segmentation-based methods in fetoscopic image registration.

Findings

Achieves higher registration accuracy than state-of-the-art methods

Enhances robustness of mosaicking in fetoscopy

Facilitates better intraoperative visualization

Abstract

In Twin-to-Twin Transfusion Syndrome (TTTS), abnormal vascular anastomoses in the monochorionic placenta can produce uneven blood flow between the two fetuses. In the current practice, TTTS is treated surgically by closing abnormal anastomoses using laser ablation. This surgery is minimally invasive and relies on fetoscopy. Limited field of view makes anastomosis identification a challenging task for the surgeon. To tackle this challenge, we propose a learning-based framework for in-vivo fetoscopy frame registration for field-of-view expansion. The novelties of this framework relies on a learning-based keypoint proposal network and an encoding strategy to filter (i) irrelevant keypoints based on fetoscopic image segmentation and (ii) inconsistent homographies. We validate of our framework on a dataset of 6 intraoperative sequences from 6 TTTS surgeries from 6 different women against the most recent state of the art algorithm, which relies on the segmentation of placenta vessels. The proposed framework achieves higher performance compared to the state of the art, paving the way for robust mosaicking to provide surgeons with context awareness during TTTS surgery.