Object Detection as Probabilistic Set Prediction

TL;DR

This paper introduces a set prediction framework for probabilistic object detection, providing a threshold-free, proper scoring rule for evaluation and training, enabling fair comparison and encouraging development of uncertainty-aware detectors.

Contribution

It proposes a novel set prediction approach using negative log-likelihood for probabilistic object detection, addressing evaluation issues and enabling fair comparisons.

Findings

Existing detectors are optimized for non-probabilistic metrics.

The proposed method can evaluate and train probabilistic detectors effectively.

Results on COCO show the need for new probabilistic detector development.

Abstract

Accurate uncertainty estimates are essential for deploying deep object detectors in safety-critical systems. The development and evaluation of probabilistic object detectors have been hindered by shortcomings in existing performance measures, which tend to involve arbitrary thresholds or limit the detector's choice of distributions. In this work, we propose to view object detection as a set prediction task where detectors predict the distribution over the set of objects. Using the negative log-likelihood for random finite sets, we present a proper scoring rule for evaluating and training probabilistic object detectors. The proposed method can be applied to existing probabilistic detectors, is free from thresholds, and enables fair comparison between architectures. Three different types of detectors are evaluated on the COCO dataset. Our results indicate that the training of existing detectors is optimized toward non-probabilistic metrics. We hope to encourage the development of new object detectors that can accurately estimate their own uncertainty. Code available at https://github.com/georghess/pmb-nll.