What's in the Black Box? The False Negative Mechanisms Inside Object Detectors

TL;DR

This paper identifies five internal failure mechanisms causing false negatives in object detectors, analyzes their prevalence in different datasets, and highlights the importance of understanding these mechanisms for robotics applications.

Contribution

It introduces a framework to quantify false negative mechanisms in object detectors and compares their occurrence across benchmark and robotics datasets.

Findings

False negative mechanisms vary significantly between datasets.

Robotics datasets exhibit different failure patterns than benchmark datasets.

Understanding these mechanisms can improve detector deployment in robotics.

Abstract

In object detection, false negatives arise when a detector fails to detect a target object. To understand why object detectors produce false negatives, we identify five 'false negative mechanisms', where each mechanism describes how a specific component inside the detector architecture failed. Focusing on two-stage and one-stage anchor-box object detector architectures, we introduce a framework for quantifying these false negative mechanisms. Using this framework, we investigate why Faster R-CNN and RetinaNet fail to detect objects in benchmark vision datasets and robotics datasets. We show that a detector's false negative mechanisms differ significantly between computer vision benchmark datasets and robotics deployment scenarios. This has implications for the translation of object detectors developed for benchmark datasets to robotics applications. Code is publicly available at https://github.com/csiro-robotics/fn_mechanisms