Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders

TL;DR

This paper introduces a sequential recurrent encoder model for land cover classification that leverages temporal Sentinel 2 data, effectively filtering clouds internally and achieving state-of-the-art accuracy with minimal preprocessing.

Contribution

The authors adapt sequence-to-sequence models with convolutional recurrent layers for EO data, enabling cloud filtering and improved land cover classification without extensive preprocessing.

Findings

Recurrent cells reduce activity during cloudy observations.

The model learns cloud-filtering schemes from data.

Achieves state-of-the-art accuracy on crop classification.

Abstract

Earth observation (EO) sensors deliver data with daily or weekly temporal resolution. Most land use and land cover (LULC) approaches, however, expect cloud-free and mono-temporal observations. The increasing temporal capabilities of today's sensors enables the use of temporal, along with spectral and spatial features. Domains, such as speech recognition or neural machine translation, work with inherently temporal data and, today, achieve impressive results using sequential encoder-decoder structures. Inspired by these sequence-to-sequence models, we adapt an encoder structure with convolutional recurrent layers in order to approximate a phenological model for vegetation classes based on a temporal sequence of Sentinel 2 (S2) images. In our experiments, we visualize internal activations over a sequence of cloudy and non-cloudy images and find several recurrent cells, which reduce the input activity for cloudy observations. Hence, we assume that our network has learned cloud-filtering schemes solely from input data, which could alleviate the need for tedious cloud-filtering as a preprocessing step for many EO approaches. Moreover, using unfiltered temporal series of top-of-atmosphere (TOA) reflectance data, we achieved in our experiments state-of-the-art classification accuracies on a large number of crop classes with minimal preprocessing compared to other classification approaches.