Estimation of Trocar and Tool Interaction Forces on the da Vinci Research Kit with Two-Step Deep Learning

TL;DR

This paper presents a two-step deep learning method to estimate external forces and torques during robotic surgery, enabling haptic feedback without additional sensors, with minimal setup time.

Contribution

It introduces a novel two-step deep learning approach for real-time force estimation in robotic surgery, compensating for mechanical effects and adapting quickly to specific setups.

Findings

Estimates external forces with a mean RMSE of 2 N.

Estimates external torques with a mean RMSE of 0.08 Nm.

Requires only about 5 minutes for intraoperative training data collection.

Abstract

Measurement of environment interaction forces during robotic minimally-invasive surgery would enable haptic feedback to the surgeon, thereby solving one long-standing limitation. Estimating this force from existing sensor data avoids the challenge of retrofitting systems with force sensors, but is difficult due to mechanical effects such as friction and compliance in the robot mechanism. We have previously shown that neural networks can be trained to estimate the internal robot joint torques, thereby enabling estimation of external forces. In this work, we extend the method to estimate external Cartesian forces and torques, and also present a two-step approach to adapt to the specific surgical setup by compensating for forces due to the interactions between the instrument shaft and cannula seal and between the trocar and patient body. Experiments show that this approach provides estimates of external forces and torques within a mean root-mean-square error (RMSE) of 2 N and 0.08 Nm, respectively. Furthermore, the two-step approach can add as little as 5 minutes to the surgery setup time, with about 4 minutes to collect intraoperative training data and 1 minute to train the second-step network.