Analysing object detectors from the perspective of co-occurring object categories

TL;DR

This paper evaluates how much state-of-the-art object detectors rely on contextual information at the category level using a masked MS COCO dataset, revealing that such dependence is generally weak but consistent when present.

Contribution

It introduces a method to measure category-level contextual dependence in object detectors and shows that this dependence is an independent property transferable across domains.

Findings

Detectors rarely rely heavily on category-level context.

When they do, detectors use similar dependency patterns.

Category-level context dependence is an independent, transferable property.

Abstract

The accuracy of state-of-the-art Faster R-CNN and YOLO object detectors are evaluated and compared on a special masked MS COCO dataset to measure how much their predictions rely on contextual information encoded at object category level. Category level representation of context is motivated by the fact that it could be an adequate way to transfer knowledge between visual and non-visual domains. According to our measurements, current detectors usually do not build strong dependency on contextual information at category level, however, when they does, they does it in a similar way, suggesting that contextual dependence of object categories is an independent property that is relevant to be transferred.