Conformal Semantic Image Segmentation: Post-hoc Quantification of Predictive Uncertainty

TL;DR

This paper introduces a lightweight, post-hoc conformal prediction method for quantifying uncertainty in semantic image segmentation, providing valid prediction sets and visualization tools to assess model confidence.

Contribution

It presents a novel conformal prediction approach tailored for semantic segmentation, enabling statistically valid uncertainty quantification without retraining models.

Findings

Valid prediction sets at specified confidence levels

Effective visualization of uncertainty via heatmaps

Demonstrated on benchmark datasets with promising results

Abstract

We propose a post-hoc, computationally lightweight method to quantify predictive uncertainty in semantic image segmentation. Our approach uses conformal prediction to generate statistically valid prediction sets that are guaranteed to include the ground-truth segmentation mask at a predefined confidence level. We introduce a novel visualization technique of conformalized predictions based on heatmaps, and provide metrics to assess their empirical validity. We demonstrate the effectiveness of our approach on well-known benchmark datasets and image segmentation prediction models, and conclude with practical insights.