Robotic Arm Platform for Multi-View Image Acquisition and 3D Reconstruction in Minimally Invasive Surgery

TL;DR

This paper presents a robotic arm platform that captures multi-view images for accurate 3D reconstruction in minimally invasive surgery, demonstrating near sub-millimeter accuracy under certain conditions.

Contribution

It introduces a robotic platform for controlled multi-view data collection in MIS, enabling improved 3D reconstruction and dataset generation for learning models.

Findings

Reconstruction accuracy is highest under operating room lighting and spherical trajectories.

The pipeline achieves an average of 1.05 mm RMS error in 3D reconstruction.

Lighting and trajectory significantly affect reconstruction quality.

Abstract

Minimally invasive surgery (MIS) offers significant benefits such as reduced recovery time and minimised patient trauma, but poses challenges in visibility and access, making accurate 3D reconstruction a significant tool in surgical planning and navigation. This work introduces a robotic arm platform for efficient multi-view image acquisition and precise 3D reconstruction in MIS settings. We adapted a laparoscope to a robotic arm and captured ex-vivo images of several ovine organs across varying lighting conditions (operating room and laparoscopic) and trajectories (spherical and laparoscopic). We employed recently released learning-based feature matchers combined with COLMAP to produce our reconstructions. The reconstructions were evaluated against high-precision laser scans for quantitative evaluation. Our results show that whilst reconstructions suffer most under realistic MIS lighting and trajectory, many versions of our pipeline achieve close to sub-millimetre accuracy with an average of 1.05 mm Root Mean Squared Error and 0.82 mm Chamfer distance. Our best reconstruction results occur with operating room lighting and spherical trajectories. Our robotic platform provides a tool for controlled, repeatable multi-view data acquisition for 3D generation in MIS environments which we hope leads to new datasets for training learning-based models.