Beyond Classification: Definition and Density-based Estimation of Calibration in Object Detection

TL;DR

This paper introduces a new way to define and estimate calibration error specifically for object detection in deep neural networks, addressing a gap in existing calibration methods.

Contribution

It adapts classification calibration concepts to object detection and proposes a consistent, differentiable estimator using kernel density estimation.

Findings

The proposed estimator outperforms existing calibration methods in experiments.

It maintains detection performance while improving calibration accuracy.

The method is applicable to structured output spaces beyond classification.

Abstract

Despite their impressive predictive performance in various computer vision tasks, deep neural networks (DNNs) tend to make overly confident predictions, which hinders their widespread use in safety-critical applications. While there have been recent attempts to calibrate DNNs, most of these efforts have primarily been focused on classification tasks, thus neglecting DNN-based object detectors. Although several recent works addressed calibration for object detection and proposed differentiable penalties, none of them are consistent estimators of established concepts in calibration. In this work, we tackle the challenge of defining and estimating calibration error specifically for this task. In particular, we adapt the definition of classification calibration error to handle the nuances associated with object detection, and predictions in structured output spaces more generally. Furthermore, we propose a consistent and differentiable estimator of the detection calibration error, utilizing kernel density estimation. Our experiments demonstrate the effectiveness of our estimator against competing train-time and post-hoc calibration methods, while maintaining similar detection performance.