Estimating Uncertainty in Landslide Segmentation Models

TL;DR

This paper evaluates methods for estimating pixel-level uncertainty in landslide segmentation models from satellite imagery, highlighting Test-Time Augmentation as the most effective approach for improving confidence assessment.

Contribution

It compares three uncertainty estimation methods without modifying model architecture, demonstrating the superior performance of Test-Time Augmentation in landslide segmentation tasks.

Findings

Test-Time Augmentation outperforms other methods in uncertainty estimation.

Uncertainty measures can improve manual review and dataset quality.

The study provides insights for robust landslide detection models.

Abstract

Landslides are a recurring, widespread hazard. Preparation and mitigation efforts can be aided by a high-quality, large-scale dataset that covers global at-risk areas. Such a dataset currently does not exist and is impossible to construct manually. Recent automated efforts focus on deep learning models for landslide segmentation (pixel labeling) from satellite imagery. However, it is also important to characterize the uncertainty or confidence levels of such segmentations. Accurate and robust uncertainty estimates can enable low-cost (in terms of manual labor) oversight of auto-generated landslide databases to resolve errors, identify hard negative examples, and increase the size of labeled training data. In this paper, we evaluate several methods for assessing pixel-level uncertainty of the segmentation. Three methods that do not require architectural changes were compared, including Pre-Threshold activations, Monte-Carlo Dropout and Test-Time Augmentation -- a method that measures the robustness of predictions in the face of data augmentation. Experimentally, the quality of the latter method was consistently higher than the others across a variety of models and metrics in our dataset.