Conformal Prediction Sets for Instance Segmentation

TL;DR

This paper introduces a conformal prediction method for instance segmentation that provides calibrated uncertainty estimates with provable guarantees, improving reliability in diverse applications like agriculture, biology, and vehicle detection.

Contribution

It develops a novel conformal prediction algorithm that generates adaptive confidence sets for instance segmentation with theoretical coverage guarantees.

Findings

Prediction sets vary in size based on query difficulty.

Achieves target coverage and outperforms existing methods.

Provides both asymptotic and finite sample guarantees.

Abstract

Current instance segmentation models achieve high performance on average predictions, but lack principled uncertainty quantification: their outputs are not calibrated, and there is no guarantee that a predicted mask is close to the ground truth. To address this limitation, we introduce a conformal prediction algorithm to generate adaptive confidence sets for instance segmentation. Given an image and a pixel coordinate query, our algorithm generates a confidence set of instance predictions for that pixel, with a provable guarantee for the probability that at least one of the predictions has high Intersection-Over-Union (IoU) with the true object instance mask. We apply our algorithm to instance segmentation examples in agricultural field delineation, cell segmentation, and vehicle detection. Empirically, we find that our prediction sets vary in size based on query difficulty and attain the target coverage, outperforming existing baselines such as Learn Then Test, Conformal Risk Control, and morphological dilation-based methods. We provide versions of the algorithm with asymptotic and finite sample guarantees.