Uncertainty evaluation of segmentation models for Earth observation

TL;DR

This paper benchmarks and evaluates uncertainty estimation methods for satellite image segmentation, focusing on their ability to identify errors and noise in remote sensing applications.

Contribution

It provides a comprehensive evaluation of existing uncertainty estimation methods specifically tailored for Earth observation segmentation models, offering practical recommendations.

Findings

Uncertainty measures can effectively identify prediction errors.

Ensemble methods outperform other approaches in uncertainty estimation.

Evaluation on diverse datasets highlights the robustness of certain methods.

Abstract

This paper investigates methods for estimating uncertainty in semantic segmentation predictions derived from satellite imagery. Estimating uncertainty for segmentation presents unique challenges compared to standard image classification, requiring scalable methods producing per-pixel estimates. While most research on this topic has focused on scene understanding or medical imaging, this work benchmarks existing methods specifically for remote sensing and Earth observation applications. Our evaluation focuses on the practical utility of uncertainty measures, testing their ability to identify prediction errors and noise-corrupted input image regions. Experiments are conducted on two remote sensing datasets, PASTIS and ForTy, selected for their differences in scale, geographic coverage, and label confidence. We perform an extensive evaluation featuring several models, such as Stochastic Segmentation Networks and ensembles, in combination with a number of neural architectures and uncertainty metrics. We make a number of practical recommendations based on our findings.