Multiresolution Fully Convolutional Networks to detect Clouds and Snow through Optical Satellite Images

TL;DR

This paper introduces a multiresolution fully convolutional neural network that fuses VNIR and SWIR satellite image data to accurately detect clouds and snow, outperforming traditional classifiers.

Contribution

It presents a novel multiresolution FCN architecture for effective cloud and snow detection by fusing VNIR and SWIR data in an end-to-end deep learning framework.

Findings

Achieved 94.31% accuracy and 97.67% F1 score for cloud detection.

Outperformed Random Forest by 30% and single-resolution FCN by 10%.

Demonstrated the potential of multi-sensor fusion in CNNs.

Abstract

Clouds and snow have similar spectral features in the visible and near-infrared (VNIR) range and are thus difficult to distinguish from each other in high resolution VNIR images. We address this issue by introducing a shortwave-infrared (SWIR) band where clouds are highly reflective, and snow is absorptive. As SWIR is typically of a lower resolution compared to VNIR, this study proposes a multiresolution fully convolutional neural network (FCN) that can effectively detect clouds and snow in VNIR images. We fuse the multiresolution bands within a deep FCN and perform semantic segmentation at the higher, VNIR resolution. Such a fusion-based classifier, trained in an end-to-end manner, achieved 94.31% overall accuracy and an F1 score of 97.67% for clouds on Resourcesat-2 data captured over the state of Uttarakhand, India. These scores were found to be 30% higher than a Random Forest classifier, and 10% higher than a standalone single-resolution FCN. Apart from being useful for cloud detection purposes, the study also highlights the potential of convolutional neural networks for multi-sensor fusion problems.