Breast Mass Detection with Faster R-CNN: On the Feasibility of Learning from Noisy Annotations

TL;DR

This paper investigates how noise in bounding box annotations affects breast mass detection using Faster R-CNN and proposes a new matching criterion to mitigate the impact of annotation noise.

Contribution

It provides a quantitative analysis of annotation noise effects and introduces a novel matching criterion to improve detection robustness in noisy training data.

Findings

Bounding box noise reduces detection performance by up to 9% in AUC.

Noise propagation occurs due to imperfect matching between ground truth and proposals.

A new matching criterion improves tolerance to annotation noise.

Abstract

In this work we study the impact of noise on the training of object detection networks for the medical domain, and how it can be mitigated by improving the training procedure. Annotating large medical datasets for training data-hungry deep learning models is expensive and time consuming. Leveraging information that is already collected in clinical practice, in the form of text reports, bookmarks or lesion measurements would substantially reduce this cost. Obtaining precise lesion bounding boxes through automatic mining procedures, however, is difficult. We provide here a quantitative evaluation of the effect of bounding box coordinate noise on the performance of Faster R-CNN object detection networks for breast mass detection. Varying degrees of noise are simulated by randomly modifying the bounding boxes: in our experiments, bounding boxes could be enlarged up to six times the original size. The noise is injected in the CBIS-DDSM collection, a well curated public mammography dataset for which accurate lesion location is available. We show how, due to an imperfect matching between the ground truth and the network bounding box proposals, the noise is propagated during training and reduces the ability of the network to correctly classify lesions from background. When using the standard Intersection over Union criterion, the area under the FROC curve decreases by up to 9%. A novel matching criterion is proposed to improve tolerance to noise.