The Box Size Confidence Bias Harms Your Object Detector

TL;DR

This paper investigates how size-related confidence bias in object detectors negatively impacts their performance, and proposes a calibration method that corrects this bias, leading to significant accuracy improvements without extra data or retraining.

Contribution

The paper formally proves the detrimental effect of size-based confidence bias on detector performance and introduces a calibration technique to mitigate this bias and enhance accuracy.

Findings

Calibration improves detector performance by up to 0.6 mAP.

Bias is present even in training data detections.

Test Time Augmentation amplifies confidence bias.

Abstract

Countless applications depend on accurate predictions with reliable confidence estimates from modern object detectors. It is well known, however, that neural networks including object detectors produce miscalibrated confidence estimates. Recent work even suggests that detectors' confidence predictions are biased with respect to object size and position, but it is still unclear how this bias relates to the performance of the affected object detectors. We formally prove that the conditional confidence bias is harming the expected performance of object detectors and empirically validate these findings. Specifically, we demonstrate how to modify the histogram binning calibration to not only avoid performance impairment but also improve performance through conditional confidence calibration. We further find that the confidence bias is also present in detections generated on the training data of the detector, which we leverage to perform our de-biasing without using additional data. Moreover, Test Time Augmentation magnifies this bias, which results in even larger performance gains from our calibration method. Finally, we validate our findings on a diverse set of object detection architectures and show improvements of up to 0.6 mAP and 0.8 mAP50 without extra data or training.