MBAPose: Mask and Bounding-Box Aware Pose Estimation of Surgical Instruments with Photorealistic Domain Randomization

TL;DR

This paper introduces MBAPose, a novel pose estimation framework for surgical instruments that leverages photorealistic domain randomization and synthetic data to improve accuracy in endoscopic images.

Contribution

The paper presents a new pose estimation model called MBAPose that enhances accuracy using synthetic training data and domain randomization techniques.

Findings

21% reduction in translation error on synthetic data

26% reduction in orientation error on synthetic data

Baseline results on real data for future research

Abstract

Surgical robots are usually controlled using a priori models based on the robots' geometric parameters, which are calibrated before the surgical procedure. One of the challenges in using robots in real surgical settings is that those parameters can change over time, consequently deteriorating control accuracy. In this context, our group has been investigating online calibration strategies without added sensors. In one step toward that goal, we have developed an algorithm to estimate the pose of the instruments' shafts in endoscopic images. In this study, we build upon that earlier work and propose a new framework to more precisely estimate the pose of a rigid surgical instrument. Our strategy is based on a novel pose estimation model called MBAPose and the use of synthetic training data. Our experiments demonstrated an improvement of 21 % for translation error and 26 % for orientation error on synthetic test data with respect to our previous work. Results with real test data provide a baseline for further research.