MultiEarth 2022 -- The Champion Solution for the Matrix Completion Challenge via Multimodal Regression and Generation

TL;DR

This paper introduces a novel multimodal regression and generation framework that effectively addresses data sparsity in satellite observations, demonstrating superior performance in the CVPR 2022 MultiEarth challenge for the Amazon Rainforest.

Contribution

It presents an adaptive real-time multimodal regression and generation approach that outperforms existing methods in satellite data completion tasks.

Findings

Achieved LPIPS of 0.2226 indicating high visual quality

Attained PSNR of 123.0372 demonstrating accurate reconstruction

Secured SSIM of 0.6347 showing good structural similarity

Abstract

Earth observation satellites have been continuously monitoring the earth environment for years at different locations and spectral bands with different modalities. Due to complex satellite sensing conditions (e.g., weather, cloud, atmosphere, orbit), some observations for certain modalities, bands, locations, and times may not be available. The MultiEarth Matrix Completion Challenge in CVPR 2022 [1] provides the multimodal satellite data for addressing such data sparsity challenges with the Amazon Rainforest as the region of interest. This work proposes an adaptive real-time multimodal regression and generation framework and achieves superior performance on unseen test queries in this challenge with an LPIPS of 0.2226, a PSNR of 123.0372, and an SSIM of 0.6347.