A surgical system for automatic registration, stiffness mapping and dynamic image overlay

TL;DR

This paper presents a surgical system that autonomously detects tumors, maps their stiffness, and overlays this information onto a 3D model using augmented reality, aiming to assist surgeons during procedures.

Contribution

It integrates registration, force sensing, and tumor localization into a unified system using advanced filtering and active level set estimation techniques.

Findings

Successful tumor detection and localization on phantom models

Effective stiffness mapping and AR overlay demonstrated

Potential to reduce surgeon cognitive load

Abstract

In this paper we develop a surgical system using the da Vinci research kit (dVRK) that is capable of autonomously searching for tumors and dynamically displaying the tumor location using augmented reality. Such a system has the potential to quickly reveal the location and shape of tumors and visually overlay that information to reduce the cognitive overload of the surgeon. We believe that our approach is one of the first to incorporate state-of-the-art methods in registration, force sensing and tumor localization into a unified surgical system. First, the preoperative model is registered to the intra-operative scene using a Bingham distribution-based filtering approach. An active level set estimation is then used to find the location and the shape of the tumors. We use a recently developed miniature force sensor to perform the palpation. The estimated stiffness map is then dynamically overlaid onto the registered preoperative model of the organ. We demonstrate the efficacy of our system by performing experiments on phantom prostate models with embedded stiff inclusions.