Soft Sampling for Robust Object Detection

TL;DR

This paper investigates the impact of missing annotations on object detection robustness and introduces Soft Sampling, a method that re-weights gradients based on overlap to improve detection performance under incomplete annotations.

Contribution

The paper proposes Soft Sampling, a novel re-weighting technique for gradients that enhances robustness of object detectors with missing annotations.

Findings

Dropping 30% of annotations only slightly reduces performance.

Soft Sampling improves detection accuracy across various annotation drop rates.

On OpenImagesV3, Soft Sampling outperforms standard baselines by over 3%.

Abstract

We study the robustness of object detection under the presence of missing annotations. In this setting, the unlabeled object instances will be treated as background, which will generate an incorrect training signal for the detector. Interestingly, we observe that after dropping 30% of the annotations (and labeling them as background), the performance of CNN-based object detectors like Faster-RCNN only drops by 5% on the PASCAL VOC dataset. We provide a detailed explanation for this result. To further bridge the performance gap, we propose a simple yet effective solution, called Soft Sampling. Soft Sampling re-weights the gradients of RoIs as a function of overlap with positive instances. This ensures that the uncertain background regions are given a smaller weight compared to the hardnegatives. Extensive experiments on curated PASCAL VOC datasets demonstrate the effectiveness of the proposed Soft Sampling method at different annotation drop rates. Finally, we show that on OpenImagesV3, which is a real-world dataset with missing annotations, Soft Sampling outperforms standard detection baselines by over 3%.