Tampered VAE for Improved Satellite Image Time Series Classification

TL;DR

This paper introduces a semi-supervised VAE framework combined with a temporal transformer model for satellite image time series classification, achieving high accuracy with less memory and limited labeled data.

Contribution

It presents a novel VAE-based semi-supervised learning approach integrated with a temporal transformer, reducing memory use and improving crop classification performance.

Findings

Superior classification accuracy with limited labeled data.

Reduced GPU memory consumption compared to spatial-temporal models.

Latent space clustering enhances interpretability.

Abstract

The unprecedented availability of spatial and temporal high-resolution satellite image time series (SITS) for crop type mapping is believed to necessitate deep learning architectures to accommodate challenges arising from both dimensions. Recent state-of-the-art deep learning models have shown promising results by stacking spatial and temporal encoders. However, we present a Pyramid Time-Series Transformer (PTST) that operates solely on the temporal dimension, i.e., neglecting the spatial dimension, can produce superior results with a drastic reduction in GPU memory consumption and easy extensibility. Furthermore, we augment it to perform semi-supervised learning by proposing a classification-friendly VAE framework that introduces clustering mechanisms into latent space and can promote linear separability therein. Consequently, a few principal axes of the latent space can explain the majority of variance in raw data. Meanwhile, the VAE framework with proposed tweaks can maintain competitive classification performance as its purely discriminative counterpart when only $40\%$ of labelled data is used. We hope the proposed framework can serve as a baseline for crop classification with SITS for its modularity and simplicity.