SAModified: A Foundation Model-Based Zero-Shot Approach for Refining Noisy Land-Use Land-Cover Maps

TL;DR

This paper introduces a zero-shot method using the Segment Anything Model to automatically refine noisy land-use land-cover maps, significantly improving classification accuracy in remote sensing applications.

Contribution

It presents a novel zero-shot approach leveraging foundation models to automatically denoise LULC maps, enhancing model performance without requiring additional labeled data.

Findings

Achieved approximately 5% improvement in downstream segmentation accuracy.

Effectively reduced label noise in LULC maps.

Demonstrated the utility of foundation models in remote sensing tasks.

Abstract

Land-use and land cover (LULC) analysis is critical in remote sensing, with wide-ranging applications across diverse fields such as agriculture, utilities, and urban planning. However, automating LULC map generation using machine learning is rendered challenging due to noisy labels. Typically, the ground truths (e.g. ESRI LULC, MapBioMass) have noisy labels that hamper the model's ability to learn to accurately classify the pixels. Further, these erroneous labels can significantly distort the performance metrics of a model, leading to misleading evaluations. Traditionally, the ambiguous labels are rectified using unsupervised algorithms. These algorithms struggle not only with scalability but also with generalization across different geographies. To overcome these challenges, we propose a zero-shot approach using the foundation model, Segment Anything Model (SAM), to automatically delineate different land parcels/regions and leverage them to relabel the unsure pixels by using the local label statistics within each detected region. We achieve a significant reduction in label noise and an improvement in the performance of the downstream segmentation model by $\approx 5\%$ when trained with denoised labels.