Real-time Nonrigid Mosaicking of Laparoscopy Images

TL;DR

This paper introduces a real-time 2D non-rigid SLAM system for laparoscopy images that compensates for tissue deformation and complex motion, enhancing surgical visualization.

Contribution

A novel 2D non-rigid SLAM system using EMDQ algorithm and GPU acceleration for real-time, dense mosaicking of deformable laparoscopy images.

Findings

Demonstrates accurate mosaicking on in vivo and synthetic data.

Achieves real-time performance with CPU and GPU parallelization.

Reduces error accumulation with an uncertainty-based loop closing method.

Abstract

The ability to extend the field of view of laparoscopy images can help the surgeons to obtain a better understanding of the anatomical context. However, due to tissue deformation, complex camera motion and significant three-dimensional (3D) anatomical surface, image pixels may have non-rigid deformation and traditional mosaicking methods cannot work robustly for laparoscopy images in real-time. To solve this problem, a novel two-dimensional (2D) non-rigid simultaneous localization and mapping (SLAM) system is proposed in this paper, which is able to compensate for the deformation of pixels and perform image mosaicking in real-time. The key algorithm of this 2D non-rigid SLAM system is the expectation maximization and dual quaternion (EMDQ) algorithm, which can generate smooth and dense deformation field from sparse and noisy image feature matches in real-time. An uncertainty-based loop closing method has been proposed to reduce the accumulative errors. To achieve real-time performance, both CPU and GPU parallel computation technologies are used for dense mosaicking of all pixels. Experimental results on \textit{in vivo} and synthetic data demonstrate the feasibility and accuracy of our non-rigid mosaicking method.