How Bad is Good enough: Noisy annotations for instrument pose estimation

TL;DR

This paper investigates how noisy annotations affect deep learning-based surgical instrument pose estimation, using synthetic radiographs with known ground truth to systematically evaluate the impact and neural networks' ability to learn from noisy data.

Contribution

It introduces a synthetic data framework for evaluating noisy annotations in pose estimation and demonstrates neural networks can learn dominant signals despite annotation noise.

Findings

Neural networks can learn dominant signals from noisy annotations with enough data.

Synthetic radiographs enable systematic evaluation of annotation noise effects.

Pose estimation accuracy is affected by the level of annotation noise.

Abstract

Though analysis of Medical Images by Deep Learning achieves unprecedented results across various applications, the effect of \emph{noisy training annotations} is rarely studied in a systematic manner. In Medical Image Analysis, most reports addressing this question concentrate on studying segmentation performance of deep learning classifiers. The absence of continuous ground truth annotations in these studies limits the value of conclusions for applications, where regression is the primary method of choice. In the application of surgical instrument pose estimation, where precision has a direct clinical impact on patient outcome, studying the effect of \emph{noisy annotations} on deep learning pose estimation techniques is of supreme importance. Real x-ray images are inadequate for this evaluation due to the unavailability of ground truth annotations. We circumvent this problem by generating synthetic radiographs, where the ground truth pose is known and therefore the pose estimation error made by the medical-expert can be estimated from experiments. Furthermore, this study shows the property of deep neural networks to learn dominant signals from noisy annotations with sufficient data in a regression setting.