Conformal Object Detection by Sequential Risk Control

TL;DR

This paper introduces Sequential Conformal Risk Control (SeqCRC), a novel method for reliable object detection with statistical guarantees, addressing safety-critical industrial applications through conformal prediction techniques.

Contribution

The paper formally defines Conformal Object Detection and proposes SeqCRC, extending conformal risk control to sequential tasks with new loss functions and a toolkit for implementation.

Findings

SeqCRC provides valid statistical guarantees for object detection.

Extensive experiments validate the effectiveness and practical trade-offs of the method.

The toolkit facilitates replication and further research in conformal object detection.

Abstract

Recent advances in object detectors have led to their adoption for industrial uses. However, their deployment in safety-critical applications is hindered by the inherent lack of reliability of neural networks and the complex structure of object detection models. To address these challenges, we turn to Conformal Prediction, a post-hoc predictive uncertainty quantification procedure with statistical guarantees that are valid for any dataset size, without requiring prior knowledge on the model or data distribution. Our contribution is manifold. First, we formally define the problem of Conformal Object Detection (COD). We introduce a novel method, Sequential Conformal Risk Control (SeqCRC), that extends the statistical guarantees of Conformal Risk Control to two sequential tasks with two parameters, as required in the COD setting. Then, we present old and new loss functions and prediction sets suited to applying SeqCRC to different cases and certification requirements. Finally, we present a conformal toolkit for replication and further exploration of our method. Using this toolkit, we perform extensive experiments that validate our approach and emphasize trade-offs and other practical consequences.