Evaluating Merging Strategies for Sampling-based Uncertainty Techniques in Object Detection

TL;DR

This paper investigates how different merging strategies affect the performance of sampling-based uncertainty estimation in object detection, highlighting the importance of affinity and clustering choices for accurate detection and uncertainty measurement.

Contribution

It provides the first comprehensive analysis of association and merging strategies in sampling-based uncertainty techniques for object detection, demonstrating their impact on detection accuracy and uncertainty quality.

Findings

Correct affinity-clustering combinations significantly improve detection performance.

Proper merging strategies enhance the reliability of uncertainty estimates.

Evaluation on diverse datasets shows robustness of proposed methods.

Abstract

There has been a recent emergence of sampling-based techniques for estimating epistemic uncertainty in deep neural networks. While these methods can be applied to classification or semantic segmentation tasks by simply averaging samples, this is not the case for object detection, where detection sample bounding boxes must be accurately associated and merged. A weak merging strategy can significantly degrade the performance of the detector and yield an unreliable uncertainty measure. This paper provides the first in-depth investigation of the effect of different association and merging strategies. We compare different combinations of three spatial and two semantic affinity measures with four clustering methods for MC Dropout with a Single Shot Multi-Box Detector. Our results show that the correct choice of affinity-clustering combination can greatly improve the effectiveness of the classification and spatial uncertainty estimation and the resulting object detection performance. We base our evaluation on a new mix of datasets that emulate near open-set conditions (semantically similar unknown classes), distant open-set conditions (semantically dissimilar unknown classes) and the common closed-set conditions (only known classes).