Realistic Data Generation for 6D Pose Estimation of Surgical Instruments

TL;DR

This paper introduces an advanced simulation environment for generating realistic surgical instrument images to improve 6D pose estimation, crucial for automation in surgical robotics, by creating large annotated datasets with high accuracy.

Contribution

We developed an automated data generation pipeline and enhanced surgical scene simulation to produce realistic datasets for surgical instrument pose estimation.

Findings

Generated 7.5k annotated images for surgical needle pose estimation.

Achieved a mean translational error of 2.59mm on occluded datasets.

Demonstrated the pipeline's effectiveness for training vision algorithms.

Abstract

Automation in surgical robotics has the potential to improve patient safety and surgical efficiency, but it is difficult to achieve due to the need for robust perception algorithms. In particular, 6D pose estimation of surgical instruments is critical to enable the automatic execution of surgical maneuvers based on visual feedback. In recent years, supervised deep learning algorithms have shown increasingly better performance at 6D pose estimation tasks; yet, their success depends on the availability of large amounts of annotated data. In household and industrial settings, synthetic data, generated with 3D computer graphics software, has been shown as an alternative to minimize annotation costs of 6D pose datasets. However, this strategy does not translate well to surgical domains as commercial graphics software have limited tools to generate images depicting realistic instrument-tissue interactions. To address these limitations, we propose an improved simulation environment for surgical robotics that enables the automatic generation of large and diverse datasets for 6D pose estimation of surgical instruments. Among the improvements, we developed an automated data generation pipeline and an improved surgical scene. To show the applicability of our system, we generated a dataset of 7.5k images with pose annotations of a surgical needle that was used to evaluate a state-of-the-art pose estimation network. The trained model obtained a mean translational error of 2.59mm on a challenging dataset that presented varying levels of occlusion. These results highlight our pipeline's success in training and evaluating novel vision algorithms for surgical robotics applications.