Impacts of Color and Texture Distortions on Earth Observation Data in Deep Learning

TL;DR

This paper investigates how color and texture distortions in Earth observation data affect deep learning model predictions, revealing a higher sensitivity to texture distortions and guiding the development of more robust models.

Contribution

It systematically analyzes the sensitivity of state-of-the-art EO segmentation models to visual distortions, highlighting their greater vulnerability to texture changes.

Findings

Models are more sensitive to texture distortions than color distortions.

Distortions significantly impact land cover classification accuracy.

Results can inform the design of more robust EO models.

Abstract

Land cover classification and change detection are two important applications of remote sensing and Earth observation (EO) that have benefited greatly from the advances of deep learning. Convolutional and transformer-based U-net models are the state-of-the-art architectures for these tasks, and their performances have been boosted by an increased availability of large-scale annotated EO datasets. However, the influence of different visual characteristics of the input EO data on a model's predictions is not well understood. In this work we systematically examine model sensitivities with respect to several color- and texture-based distortions on the input EO data during inference, given models that have been trained without such distortions. We conduct experiments with multiple state-of-the-art segmentation networks for land cover classification and show that they are in general more sensitive to texture than to color distortions. Beyond revealing intriguing characteristics of widely used land cover classification models, our results can also be used to guide the development of more robust models within the EO domain.