EasyVis2: A Real Time Multi-view 3D Visualization System for Laparoscopic Surgery Training Enhanced by a Deep Neural Network YOLOv8-Pose

TL;DR

EasyVis2 is a real-time 3D visualization system for laparoscopic surgery that uses micro-cameras and a deep neural network to accurately estimate instrument positions and enhance surgical training.

Contribution

The paper introduces a novel multi-view 3D visualization system with an adapted YOLOv8-Pose neural network for improved accuracy and efficiency in laparoscopic surgery visualization.

Findings

Enhanced 3D reconstruction accuracy with the new system

Reduced computation time compared to previous methods

High accuracy in 2D pose estimation

Abstract

EasyVis2 is a system designed to provide hands-free, real-time 3D visualization for laparoscopic surgery. It incorporates a surgical trocar equipped with an array of micro-cameras, which can be inserted into the body cavity to offer an enhanced field of view and a 3D perspective of the surgical procedure. A specialized deep neural network algorithm, YOLOv8-Pose, is utilized to estimate the position and orientation of surgical instruments in each individual camera view. These multi-view estimates enable the calculation of 3D poses of surgical tools, facilitating the rendering of a 3D surface model of the instruments, overlaid on the background scene, for real-time visualization. This study presents methods for adapting YOLOv8-Pose to the EasyVis2 system, including the development of a tailored training dataset. Experimental results demonstrate that, with an identical number of cameras, the new system improves 3D reconstruction accuracy and reduces computation time. Additionally, the adapted YOLOv8-Pose system shows high accuracy in 2D pose estimation.