Spectral Synthesis for Satellite-to-Satellite Translation

TL;DR

This paper introduces a novel unsupervised spectral synthesis method for satellite imagery that enhances data consistency across different sensors and improves downstream tasks like cloud detection.

Contribution

It proposes a new shared spectral reconstruction loss for unsupervised image translation, enabling synthetic spectral band generation across multispectral satellite sensors.

Findings

Cross-domain reconstruction outperforms second vantage point measurements.

Synthetic bands improve cloud detection segmentation performance.

Method enables more homogeneous multispectral datasets.

Abstract

Earth observing satellites carrying multi-spectral sensors are widely used to monitor the physical and biological states of the atmosphere, land, and oceans. These satellites have different vantage points above the earth and different spectral imaging bands resulting in inconsistent imagery from one to another. This presents challenges in building downstream applications. What if we could generate synthetic bands for existing satellites from the union of all domains? We tackle the problem of generating synthetic spectral imagery for multispectral sensors as an unsupervised image-to-image translation problem with partial labels and introduce a novel shared spectral reconstruction loss. Simulated experiments performed by dropping one or more spectral bands show that cross-domain reconstruction outperforms measurements obtained from a second vantage point. On a downstream cloud detection task, we show that generating synthetic bands with our model improves segmentation performance beyond our baseline. Our proposed approach enables synchronization of multispectral data and provides a basis for more homogeneous remote sensing datasets.